Jul 30 12:59:36.622873 ip-10-0-143-33 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 12:59:36.969643 ip-10-0-143-33 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 12:59:36.969643 ip-10-0-143-33 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 12:59:36.969643 ip-10-0-143-33 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 12:59:36.969643 ip-10-0-143-33 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 12:59:36.969643 ip-10-0-143-33 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 12:59:36.971691 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.971591    2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 12:59:36.976389 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976250    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 12:59:36.976389 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976385    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 12:59:36.976389 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976389    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 12:59:36.976389 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976393    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 12:59:36.976389 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976396    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976400    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976403    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976406    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976409    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976412    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976415    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976417    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976420    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976423    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976428    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976431    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976434    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976437    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976439    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976442    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976444    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976447    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976449    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976452    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 12:59:36.976586 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976454    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976457    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976460    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976463    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976470    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976473    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976476    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976479    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976481    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976484    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976486    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976489    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976491    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976495    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976498    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976501    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976503    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976506    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976509    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976512    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 12:59:36.977082 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976514    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976517    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976520    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976522    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976525    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976527    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976530    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976533    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976535    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976541    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976543    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976546    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976548    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976551    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976553    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976556    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976558    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976561    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976564    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976566    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 12:59:36.977567 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976569    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976571    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976574    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976578    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976582    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976586    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976589    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976605    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976609    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976611    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976614    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976617    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976620    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976623    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976626    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976628    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976631    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976633    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976636    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976645    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 12:59:36.978094 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976648    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976650    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976653    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976656    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976658    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976664    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976668    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976671    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976673    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976676    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976678    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976681    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976685    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976688    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.976691    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977445    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977454    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977458    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977462    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 12:59:36.978584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977465    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977468    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977471    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977474    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977477    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977481    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977484    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977487    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977489    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977491    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977494    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977497    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977499    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977503    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977505    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977508    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977510    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977513    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977516    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977518    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 12:59:36.979064 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977521    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977523    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977526    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977528    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977531    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977533    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977536    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977538    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977541    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977543    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977546    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977549    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977551    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977554    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977556    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977559    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977562    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977564    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977568    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977571    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 12:59:36.979554 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977573    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977576    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977579    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977581    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977584    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977587    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977589    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977592    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977609    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977612    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977614    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977617    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977620    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977622    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977625    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977628    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977630    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977633    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977635    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977639    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 12:59:36.980089 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977643    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977646    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977650    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977654    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977657    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977660    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977663    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977666    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977669    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977672    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977674    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977677    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977679    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977682    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977684    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977687    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977689    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977692    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977694    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 12:59:36.980582 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977698    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977700    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977703    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977707    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977710    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977712    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977715    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977718    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977720    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977722    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977725    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977728    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977731    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977733    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977736    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.977739    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978110    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978115    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978117    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 12:59:36.981061 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978120    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978123    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978126    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978128    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978131    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978134    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978137    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978139    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978142    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978145    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978147    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978150    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978153    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978156    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978158    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978174    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978178    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978181    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978184    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978188    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 12:59:36.981527 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978190    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978193    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978196    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978199    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978202    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978205    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978208    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978210    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978213    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978216    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978219    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978222    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978225    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978228    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978230    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978233    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978236    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978238    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978240    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978243    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 12:59:36.982019 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978245    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978248    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978251    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978253    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978256    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978258    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978261    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978263    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978266    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978268    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978271    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978274    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978277    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978281    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978283    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978286    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978289    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978292    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978294    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978297    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 12:59:36.982584 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978300    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978302    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978305    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978307    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978310    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978312    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978315    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978317    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978320    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978323    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978325    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978328    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978330    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978333    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978335    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978338    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978340    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978343    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978345    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978348    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 12:59:36.983099 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978351    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978354    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978357    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978359    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978362    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978364    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978367    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978369    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978372    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978375    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978377    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978380    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978382    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978388    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978392    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.978395    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978463    2557 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978470    2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978477    2557 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978481    2557 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 12:59:36.983578 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978486    2557 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978489    2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978493    2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978497    2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978501    2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978504    2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978507    2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978510    2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978513    2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978516    2557 flags.go:64] FLAG: --cgroup-root=""
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978519    2557 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978522    2557 flags.go:64] FLAG: --client-ca-file=""
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978526    2557 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978529    2557 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978534    2557 flags.go:64] FLAG: --cluster-domain=""
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978536    2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978539    2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978542    2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978546    2557 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978550    2557 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978553    2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978556    2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978560    2557 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978563    2557 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 12:59:36.984110 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978566    2557 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978571    2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978574    2557 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978577    2557 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978581    2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978584    2557 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978587    2557 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978590    2557 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978605    2557 flags.go:64] FLAG: --enable-server="true"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978609    2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978614    2557 flags.go:64] FLAG: --event-burst="100"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978617    2557 flags.go:64] FLAG: --event-qps="50"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978620    2557 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978624    2557 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978626    2557 flags.go:64] FLAG: --eviction-hard=""
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978630    2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978633    2557 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978636    2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978640    2557 flags.go:64] FLAG: --eviction-soft=""
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978643    2557 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978646    2557 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978649    2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978652    2557 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978655    2557 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978658    2557 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 12:59:36.984711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978661    2557 flags.go:64] FLAG: --feature-gates=""
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978665    2557 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978668    2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978671    2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978675    2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978678    2557 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978681    2557 flags.go:64] FLAG: --help="false"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978684    2557 flags.go:64] FLAG: --hostname-override="ip-10-0-143-33.ec2.internal"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978688    2557 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978691    2557 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978693    2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978697    2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978701    2557 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978704    2557 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978706    2557 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978709    2557 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978712    2557 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978715    2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978718    2557 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978721    2557 flags.go:64] FLAG: --kube-reserved=""
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978725    2557 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978728    2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978731    2557 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978734    2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 12:59:36.985338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978737    2557 flags.go:64] FLAG: --lock-file=""
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978739    2557 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978742    2557 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978745    2557 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978751    2557 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978754    2557 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978757    2557 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978760    2557 flags.go:64] FLAG: --logging-format="text"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978763    2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978766    2557 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978769    2557 flags.go:64] FLAG: --manifest-url=""
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978772    2557 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978776    2557 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978779    2557 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978783    2557 flags.go:64] FLAG: --max-pods="110"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978786    2557 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978789    2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978792    2557 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978795    2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978798    2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978801    2557 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978804    2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978811    2557 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978814    2557 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978817    2557 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 12:59:36.986036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978820    2557 flags.go:64] FLAG: --pod-cidr=""
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978823    2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978828    2557 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978831    2557 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978836    2557 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978839    2557 flags.go:64] FLAG: --port="10250"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978842    2557 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978845    2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-043273525110f6786"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978848    2557 flags.go:64] FLAG: --qos-reserved=""
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978851    2557 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978854    2557 flags.go:64] FLAG: --register-node="true"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978857    2557 flags.go:64] FLAG: --register-with-taints=""
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978861    2557 flags.go:64] FLAG: --registry-burst="10"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978864    2557 flags.go:64] FLAG: --registry-qps="5"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978867    2557 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978870    2557 flags.go:64] FLAG: --reserved-memory=""
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978874    2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978877    2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978880    2557 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978883    2557 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978886    2557 flags.go:64] FLAG: --runonce="false"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978889    2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978892    2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978894    2557 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978897    2557 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 12:59:36.986694 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978900    2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978903    2557 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978906    2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978909    2557 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978912    2557 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978915    2557 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978918    2557 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978920    2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978923    2557 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978926    2557 flags.go:64] FLAG: --system-cgroups=""
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978929    2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978936    2557 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978939    2557 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978942    2557 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978946    2557 flags.go:64] FLAG: --tls-min-version=""
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978949    2557 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978952    2557 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978955    2557 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978958    2557 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978961    2557 flags.go:64] FLAG: --v="2"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978965    2557 flags.go:64] FLAG: --version="false"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978970    2557 flags.go:64] FLAG: --vmodule=""
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978973    2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.978977    2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979104    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 12:59:36.987306 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979108    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979111    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979114    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979117    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979120    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979122    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979125    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979128    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979130    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979133    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979135    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979138    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979140    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979143    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979145    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979148    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979152    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979155    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979160    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 12:59:36.987990 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979163    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979165    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979168    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979172    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979174    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979177    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979180    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979183    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979185    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979188    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979192    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979194    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979197    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979200    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979202    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979205    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979208    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979210    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979212    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979215    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 12:59:36.988464 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979217    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979221    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979225    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979228    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979231    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979233    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979236    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979239    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979242    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979244    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979247    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979251    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979253    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979256    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979259    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979263    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979266    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979268    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979271    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 12:59:36.988961 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979274    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979276    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979279    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979281    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979287    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979290    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979292    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979295    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979297    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979300    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979303    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979305    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979308    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979311    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979313    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979316    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979318    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979321    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979323    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979325    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 12:59:36.989421 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979328    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979331    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979333    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979336    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979340    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979342    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979345    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979347    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979351    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979354    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979357    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979359    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979363    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979366    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979369    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979372    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979375    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979378    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979381    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 12:59:36.989938 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.979384    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 12:59:36.990388 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.979389    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 12:59:36.990388 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.986837    2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 12:59:36.990388 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.986855    2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 12:59:36.990388 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.987491    2557 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 12:59:36.990388 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.987501    2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 12:59:36.990388 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987573    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 12:59:36.990388 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987579    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 12:59:36.990388 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987583    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 12:59:36.990388 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987587    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 12:59:36.990388 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987590    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 12:59:36.990388 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987605    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 12:59:36.990388 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987609    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 12:59:36.990388 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987612    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 12:59:36.990388 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987614    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987618    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987620    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987623    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987626    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987629    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987631    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987634    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987637    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987640    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987642    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987646    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987650    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987653    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987656    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987659    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987662    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987665    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987668    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 12:59:36.990771 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987671    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987674    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987678    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987681    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987684    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987687    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987690    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987692    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987695    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987699    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987703    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987706    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987710    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987714    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987717    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987720    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987723    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987726    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987729    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 12:59:36.991236 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987732    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987735    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987738    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987740    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987743    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987746    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987749    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987752    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987754    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987757    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987760    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987762    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987765    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987768    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987770    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987773    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987783    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987786    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987789    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987791    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 12:59:36.991773 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987794    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987797    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987799    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987802    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987805    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987807    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987810    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987813    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987815    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987818    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987820    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987823    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987826    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987828    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987831    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987833    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987836    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987839    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987841    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987844    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 12:59:36.992293 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987846    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987849    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987851    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987854    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987856    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987859    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987862    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987865    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987868    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987877    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987881    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987883    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.987886    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.987891    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988047    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 12:59:36.992790 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988052    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988055    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988058    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988067    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988070    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988073    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988075    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988078    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988081    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988085    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988089    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988092    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988095    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988098    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988100    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988103    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988105    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988108    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988111    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 12:59:36.993161 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988113    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988116    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988118    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988121    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988123    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988126    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988129    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988132    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988141    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988143    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988146    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988148    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988151    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988153    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988156    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988160    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988164    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988166    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988169    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 12:59:36.993644 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988172    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988174    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988177    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988179    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988182    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988184    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988187    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988189    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988192    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988194    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988197    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988199    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988202    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988204    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988216    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988219    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988221    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988224    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988227    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988230    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 12:59:36.994110 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988232    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988235    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988244    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988247    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988250    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988253    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988255    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988258    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988260    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988263    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988266    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988270    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988273    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988276    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988279    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988281    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988284    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988287    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988289    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 12:59:36.994576 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988292    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988295    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988297    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988300    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988302    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988305    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988307    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988310    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988312    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988315    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988317    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988322    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988325    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988328    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988331    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988333    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988342    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988345    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988348    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988350    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 12:59:36.995081 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:36.988353    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 12:59:36.995630 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.988358    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 12:59:36.995630 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.988480    2557 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 12:59:36.995630 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.990343    2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 12:59:36.995630 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.991564    2557 server.go:1019] "Starting client certificate rotation"
Jul 30 12:59:36.995630 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.991667    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 12:59:36.995630 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.991709    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 12:59:36.995630 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.993885    2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 12:59:36.998421 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:36.998398    2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 12:59:37.009361 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.009345    2557 log.go:25] "Validated CRI v1 runtime API"
Jul 30 12:59:37.011909 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.011887    2557 log.go:25] "Validated CRI v1 image API"
Jul 30 12:59:37.014196 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.014179    2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 12:59:37.017023 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.017003    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 12:59:37.017944 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.017919    2557 fs.go:135] Filesystem UUIDs: map[0464bb49-aa52-44b6-9cd7-a590c7e46745:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bfd02761-6cf7-419d-96fc-33d09407be06:/dev/nvme0n1p3]
Jul 30 12:59:37.018022 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.017942    2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 12:59:37.023111 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.023004    2557 manager.go:216] Machine: {Timestamp:2026-07-30 12:59:37.021865832 +0000 UTC m=+0.294330240 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100585 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bfa492286acd119c95c9d50a224d8 SystemUUID:ec2bfa49-2286-acd1-19c9-5c9d50a224d8 BootID:ab35eb42-6a6d-4fe7-bd86-2a94aff06542 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:df:35:70:24:2d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:df:35:70:24:2d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:e2:f0:cb:be:44 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 12:59:37.023111 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.023108    2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 12:59:37.023216 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.023182    2557 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 12:59:37.025333 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.025309    2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 12:59:37.025494 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.025334    2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-33.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 12:59:37.026037 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.026027    2557 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 12:59:37.026068 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.026040    2557 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 12:59:37.026068 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.026047    2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 12:59:37.027193 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.027182    2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 12:59:37.027946 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.027937    2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 12:59:37.029587 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.029576    2557 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 12:59:37.029727 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.029716    2557 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 12:59:37.029767 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.029751    2557 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 12:59:37.029809 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.029768    2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 12:59:37.029809 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.029782    2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 12:59:37.029809 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.029791    2557 kubelet.go:398] "Adding apiserver pod source"
Jul 30 12:59:37.029809 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.029801    2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 12:59:37.030185 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.030171    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 12:59:37.030218 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.030191    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 12:59:37.032558 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.032538    2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 12:59:37.035208 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.035193    2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 12:59:37.035272 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.035222    2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 12:59:37.036194 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.036176    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ptdh5"
Jul 30 12:59:37.036545 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.036534    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 12:59:37.036588 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.036551    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 12:59:37.036588 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.036557    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 12:59:37.036588 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.036562    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 12:59:37.036588 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.036574    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 12:59:37.036588 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.036581    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 12:59:37.036736 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.036591    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 12:59:37.036736 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.036611    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 12:59:37.036736 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.036618    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 12:59:37.036736 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.036623    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 12:59:37.036736 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.036636    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 12:59:37.036736 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.036641    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 12:59:37.037929 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.037914    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 12:59:37.037929 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.037928    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 12:59:37.041119 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.040924    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 12:59:37.041119 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.040925    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-33.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 12:59:37.041119 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.040962    2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-33.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 12:59:37.041641 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.041619    2557 server.go:1268] "Started kubelet"
Jul 30 12:59:37.042272 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.042242    2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 12:59:37.042468 ip-10-0-143-33 systemd[1]: Started Kubernetes Kubelet.
Jul 30 12:59:37.042705 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.042413    2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 12:59:37.042705 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.042513    2557 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 12:59:37.043789 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.043755    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ptdh5"
Jul 30 12:59:37.044076 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.044053    2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 12:59:37.045441 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.045426    2557 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 12:59:37.047889 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.047859    2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 12:59:37.051435 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.051419    2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 12:59:37.051435 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.051427    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 12:59:37.051559 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.051497    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-33.ec2.internal\" not found"
Jul 30 12:59:37.051991 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.051960    2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 12:59:37.051991 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.051961    2557 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 12:59:37.052097 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.051998    2557 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 12:59:37.052097 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.052078    2557 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 12:59:37.052097 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.052087    2557 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 12:59:37.052842 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.052824    2557 factory.go:55] Registering systemd factory
Jul 30 12:59:37.052842 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.052843    2557 factory.go:223] Registration of the systemd container factory successfully
Jul 30 12:59:37.053096 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.053081    2557 factory.go:153] Registering CRI-O factory
Jul 30 12:59:37.053096 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.053097    2557 factory.go:223] Registration of the crio container factory successfully
Jul 30 12:59:37.053203 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.053136    2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 12:59:37.053203 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.053156    2557 factory.go:103] Registering Raw factory
Jul 30 12:59:37.053203 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.053168    2557 manager.go:1196] Started watching for new ooms in manager
Jul 30 12:59:37.053468 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.053457    2557 manager.go:349] Starting recovery of all containers
Jul 30 12:59:37.058211 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.058188    2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 12:59:37.060550 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.060529    2557 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-33.ec2.internal\" not found" node="ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.061131 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.061107    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 12:59:37.064409 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.064308    2557 manager.go:354] Recovery completed
Jul 30 12:59:37.068379 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.068367    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 12:59:37.069949 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.069935    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 12:59:37.069988 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.069961    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 12:59:37.069988 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.069971    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasSufficientPID"
Jul 30 12:59:37.070481 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.070466    2557 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 12:59:37.070481 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.070479    2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 12:59:37.070586 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.070485    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 12:59:37.070586 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.070495    2557 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 12:59:37.071152 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.071137    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 12:59:37.071217 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.071161    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 12:59:37.071217 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.071170    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasSufficientPID"
Jul 30 12:59:37.072148 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.072134    2557 policy_none.go:49] "None policy: Start"
Jul 30 12:59:37.072224 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.072153    2557 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 12:59:37.072224 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.072167    2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 12:59:37.073065 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.073053    2557 policy_none.go:47] "Start"
Jul 30 12:59:37.108274 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.108129    2557 manager.go:337] "Starting Device Plugin manager"
Jul 30 12:59:37.108362 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.108302    2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 12:59:37.108362 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.108317    2557 server.go:87] "Starting device plugin registration server"
Jul 30 12:59:37.108556 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.108546    2557 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 12:59:37.108627 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.108558    2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 12:59:37.108753 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.108732    2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 12:59:37.108865 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.108826    2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 12:59:37.108865 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.108840    2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 12:59:37.109335 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.109316    2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 12:59:37.109416 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.109369    2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-33.ec2.internal\" not found"
Jul 30 12:59:37.177025 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.177005    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 12:59:37.177143 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.177049    2557 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 12:59:37.177143 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.177069    2557 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 12:59:37.177143 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.177102    2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 12:59:37.179295 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.179272    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 12:59:37.209582 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.209549    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 12:59:37.210425 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.210408    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 12:59:37.210499 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.210440    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 12:59:37.210499 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.210451    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasSufficientPID"
Jul 30 12:59:37.210499 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.210477    2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.220000 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.219938    2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.220000 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.219961    2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-33.ec2.internal\": node \"ip-10-0-143-33.ec2.internal\" not found"
Jul 30 12:59:37.236525 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.236501    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-33.ec2.internal\" not found"
Jul 30 12:59:37.278650 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.278595    2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-33.ec2.internal"]
Jul 30 12:59:37.278755 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.278697    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 12:59:37.280431 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.280415    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 12:59:37.280506 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.280444    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 12:59:37.280506 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.280454    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasSufficientPID"
Jul 30 12:59:37.281873 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.281861    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 12:59:37.282012 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.281994    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.282070 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.282027    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 12:59:37.282679 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.282661    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 12:59:37.282754 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.282687    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 12:59:37.282754 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.282697    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasSufficientPID"
Jul 30 12:59:37.282835 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.282762    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 12:59:37.282835 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.282790    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 12:59:37.282835 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.282805    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasSufficientPID"
Jul 30 12:59:37.283914 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.283899    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.283966 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.283926    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 12:59:37.284760 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.284743    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 12:59:37.284852 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.284769    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 12:59:37.284852 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.284781    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeHasSufficientPID"
Jul 30 12:59:37.315590 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.315568    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-33.ec2.internal\" not found" node="ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.320409 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.320392    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-33.ec2.internal\" not found" node="ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.337625 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.337593    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-33.ec2.internal\" not found"
Jul 30 12:59:37.353914 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.353897    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5239ea0e976e2f1f2ffc2d6a67116fe9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal\" (UID: \"5239ea0e976e2f1f2ffc2d6a67116fe9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.354002 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.353931    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5239ea0e976e2f1f2ffc2d6a67116fe9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal\" (UID: \"5239ea0e976e2f1f2ffc2d6a67116fe9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.354002 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.353956    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd398727194bd86b778ce466d77f4b02-config\") pod \"kube-apiserver-proxy-ip-10-0-143-33.ec2.internal\" (UID: \"bd398727194bd86b778ce466d77f4b02\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.438072 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.438043    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-33.ec2.internal\" not found"
Jul 30 12:59:37.454875 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.454844    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5239ea0e976e2f1f2ffc2d6a67116fe9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal\" (UID: \"5239ea0e976e2f1f2ffc2d6a67116fe9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.454875 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.454863    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5239ea0e976e2f1f2ffc2d6a67116fe9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal\" (UID: \"5239ea0e976e2f1f2ffc2d6a67116fe9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.455049 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.454901    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5239ea0e976e2f1f2ffc2d6a67116fe9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal\" (UID: \"5239ea0e976e2f1f2ffc2d6a67116fe9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.455049 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.454926    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd398727194bd86b778ce466d77f4b02-config\") pod \"kube-apiserver-proxy-ip-10-0-143-33.ec2.internal\" (UID: \"bd398727194bd86b778ce466d77f4b02\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.455049 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.454976    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd398727194bd86b778ce466d77f4b02-config\") pod \"kube-apiserver-proxy-ip-10-0-143-33.ec2.internal\" (UID: \"bd398727194bd86b778ce466d77f4b02\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.455049 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.454998    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5239ea0e976e2f1f2ffc2d6a67116fe9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal\" (UID: \"5239ea0e976e2f1f2ffc2d6a67116fe9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.539294 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.539199    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-33.ec2.internal\" not found"
Jul 30 12:59:37.622652 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.622626    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.623783 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.623757    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:37.640018 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.639996    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-33.ec2.internal\" not found"
Jul 30 12:59:37.746160 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.746106    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-33.ec2.internal\" not found"
Jul 30 12:59:37.853838 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.853747    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-33.ec2.internal\" not found"
Jul 30 12:59:37.953908 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:37.953871    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-33.ec2.internal\" not found"
Jul 30 12:59:37.993063 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.993034    2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 12:59:37.993512 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.993188    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 12:59:37.993512 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:37.993201    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 12:59:38.025982 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.025960    2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 12:59:38.030635 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.030617    2557 apiserver.go:52] "Watching apiserver"
Jul 30 12:59:38.041697 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.041674    2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 12:59:38.042049 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.042028    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-74gmj","openshift-network-diagnostics/network-check-target-769q2","openshift-cluster-node-tuning-operator/tuned-htmgz","openshift-dns/node-resolver-fng7k","openshift-image-registry/node-ca-5h277","openshift-multus/multus-additional-cni-plugins-7mbgz","openshift-multus/multus-pmvrj","openshift-network-operator/iptables-alerter-fczjl","openshift-ovn-kubernetes/ovnkube-node-4q7bg","kube-system/global-pull-secret-syncer-qbp52","kube-system/konnectivity-agent-hn766","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"]
Jul 30 12:59:38.043443 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.043428    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:38.043510 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.043492    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 12:59:38.044404 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.044387    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:38.044485 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.044464    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 12:59:38.045760 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.045733    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.048163 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.048099    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 12:54:37 +0000 UTC" deadline="2028-01-20 08:36:54.6511783 +0000 UTC"
Jul 30 12:59:38.048163 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.048160    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12931h37m16.603024109s"
Jul 30 12:59:38.048334 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.048195    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fng7k"
Jul 30 12:59:38.049140 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.049123    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cvrlb\""
Jul 30 12:59:38.049309 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.049276    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 12:59:38.049421 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.049340    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 12:59:38.049859 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.049840    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5h277"
Jul 30 12:59:38.050783 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.050769    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ssztn\""
Jul 30 12:59:38.050954 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.050936    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 12:59:38.051106 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.051085    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.051202 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.051186    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 12:59:38.051568 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.051554    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:38.051720 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.051558    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 12:59:38.052147 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.052127    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.052280 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.052266    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 12:59:38.052767 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.052750    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 12:59:38.052895 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.052881    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 12:59:38.053063 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.053043    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f5b9b\""
Jul 30 12:59:38.053264 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.053251    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fczjl"
Jul 30 12:59:38.054102 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.054082    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 12:59:38.054187 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.054175    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 12:59:38.054363 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.054347    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 12:59:38.054438 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.054411    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mzt7l\""
Jul 30 12:59:38.054496 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.054446    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 12:59:38.054553 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.054536    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 12:59:38.054723 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.054704    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.055319 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.055302    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 12:59:38.055410 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.055329    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gw6dk\""
Jul 30 12:59:38.055985 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.055967    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:38.056068 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.056049    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 12:59:38.056106 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.056058    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gfj8m\""
Jul 30 12:59:38.056322 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.056309    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 12:59:38.056972 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.056958    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 12:59:38.057293 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.057278    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hn766"
Jul 30 12:59:38.057489 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.057470    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 12:59:38.058472 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.057928    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 12:59:38.058472 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.057977    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 12:59:38.058472 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.057997    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 12:59:38.058472 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058248    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 12:59:38.058472 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058360    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 12:59:38.058472 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058357    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-lib-modules\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.058472 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058398    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5972f608-8acf-4650-8b1e-5ad1d2d49bed-cnibin\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.058472 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058416    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5972f608-8acf-4650-8b1e-5ad1d2d49bed-os-release\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.058472 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058456    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5972f608-8acf-4650-8b1e-5ad1d2d49bed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.058472 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058474    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5972f608-8acf-4650-8b1e-5ad1d2d49bed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058490    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9rzq\" (UniqueName: \"kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq\") pod \"network-check-target-769q2\" (UID: \"5f461854-f3bc-4026-8c6b-f2b6c756dd17\") " pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058510    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-run-netns\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058491    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bsvnp\""
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058560    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e9b91626-80d1-487b-bdf6-75b892b2dd6b-serviceca\") pod \"node-ca-5h277\" (UID: \"e9b91626-80d1-487b-bdf6-75b892b2dd6b\") " pod="openshift-image-registry/node-ca-5h277"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058592    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-sysconfig\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058634    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5972f608-8acf-4650-8b1e-5ad1d2d49bed-system-cni-dir\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058662    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5972f608-8acf-4650-8b1e-5ad1d2d49bed-cni-binary-copy\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058670    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058696    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-var-lib-cni-bin\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058719    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-multus-conf-dir\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058735    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-etc-kubernetes\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058751    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcg82\" (UniqueName: \"kubernetes.io/projected/9805b510-a97b-432c-9219-51bea9f4f08f-kube-api-access-gcg82\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058787    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bad62a71-0b34-453b-97c5-bb6ef94cabda-hosts-file\") pod \"node-resolver-fng7k\" (UID: \"bad62a71-0b34-453b-97c5-bb6ef94cabda\") " pod="openshift-dns/node-resolver-fng7k"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058795    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058809    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9b91626-80d1-487b-bdf6-75b892b2dd6b-host\") pod \"node-ca-5h277\" (UID: \"e9b91626-80d1-487b-bdf6-75b892b2dd6b\") " pod="openshift-image-registry/node-ca-5h277"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058837    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64l85\" (UniqueName: \"kubernetes.io/projected/e9b91626-80d1-487b-bdf6-75b892b2dd6b-kube-api-access-64l85\") pod \"node-ca-5h277\" (UID: \"e9b91626-80d1-487b-bdf6-75b892b2dd6b\") " pod="openshift-image-registry/node-ca-5h277"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058883    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-var-lib-cni-multus\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.059027 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058920    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-var-lib-kubelet\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058937    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-hostroot\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058983    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ae0ce23-6304-4f04-b3a4-c7e3497aa75b-host-slash\") pod \"iptables-alerter-fczjl\" (UID: \"6ae0ce23-6304-4f04-b3a4-c7e3497aa75b\") " pod="openshift-network-operator/iptables-alerter-fczjl"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.058999    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-sysctl-conf\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059029    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-systemd\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059044    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85235635-3099-4830-a013-38009fe3b04f-tmp\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059066    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5972f608-8acf-4650-8b1e-5ad1d2d49bed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059088    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5tvx\" (UniqueName: \"kubernetes.io/projected/5972f608-8acf-4650-8b1e-5ad1d2d49bed-kube-api-access-h5tvx\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059103    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-system-cni-dir\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059118    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-run-k8s-cni-cncf-io\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059132    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9805b510-a97b-432c-9219-51bea9f4f08f-multus-daemon-config\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059181    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8t5f\" (UniqueName: \"kubernetes.io/projected/6ae0ce23-6304-4f04-b3a4-c7e3497aa75b-kube-api-access-t8t5f\") pod \"iptables-alerter-fczjl\" (UID: \"6ae0ce23-6304-4f04-b3a4-c7e3497aa75b\") " pod="openshift-network-operator/iptables-alerter-fczjl"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059205    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs\") pod \"network-metrics-daemon-74gmj\" (UID: \"c083abff-9168-456b-ba24-dd906dc138b2\") " pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059241    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfg7c\" (UniqueName: \"kubernetes.io/projected/bad62a71-0b34-453b-97c5-bb6ef94cabda-kube-api-access-mfg7c\") pod \"node-resolver-fng7k\" (UID: \"bad62a71-0b34-453b-97c5-bb6ef94cabda\") " pod="openshift-dns/node-resolver-fng7k"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059268    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-cnibin\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059290    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-multus-socket-dir-parent\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.059700 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059344    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6ae0ce23-6304-4f04-b3a4-c7e3497aa75b-iptables-alerter-script\") pod \"iptables-alerter-fczjl\" (UID: \"6ae0ce23-6304-4f04-b3a4-c7e3497aa75b\") " pod="openshift-network-operator/iptables-alerter-fczjl"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059366    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-modprobe-d\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059382    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-kubernetes\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059396    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-sys\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059410    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-var-lib-kubelet\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059432    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-host\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059478    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-multus-cni-dir\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059513    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-os-release\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059536    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9805b510-a97b-432c-9219-51bea9f4f08f-cni-binary-copy\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059554    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7nm8\" (UniqueName: \"kubernetes.io/projected/c083abff-9168-456b-ba24-dd906dc138b2-kube-api-access-d7nm8\") pod \"network-metrics-daemon-74gmj\" (UID: \"c083abff-9168-456b-ba24-dd906dc138b2\") " pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059591    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bad62a71-0b34-453b-97c5-bb6ef94cabda-tmp-dir\") pod \"node-resolver-fng7k\" (UID: \"bad62a71-0b34-453b-97c5-bb6ef94cabda\") " pod="openshift-dns/node-resolver-fng7k"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059614    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-sysctl-d\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059669    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/85235635-3099-4830-a013-38009fe3b04f-etc-tuned\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059683    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcwv5\" (UniqueName: \"kubernetes.io/projected/85235635-3099-4830-a013-38009fe3b04f-kube-api-access-fcwv5\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059707    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-run-multus-certs\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059743    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-run\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.059988    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-f5n7z\""
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.060106    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 12:59:38.060341 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.060136    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 12:59:38.061582 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.061564    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 12:59:38.061677 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.061645    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-72zcs\""
Jul 30 12:59:38.061723 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.061693    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 12:59:38.061847 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.061834    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 12:59:38.065328 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.065304    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 12:59:38.065446 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.065398    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal"
Jul 30 12:59:38.067669 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.067630    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-33.ec2.internal"]
Jul 30 12:59:38.067755 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.067736    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 12:59:38.077301 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.077282    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 12:59:38.077472 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.077452    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal"]
Jul 30 12:59:38.094095 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.094078    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vvhc4"
Jul 30 12:59:38.103988 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.103929    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vvhc4"
Jul 30 12:59:38.146338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.146317    2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 12:59:38.153507 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.153487    2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 12:59:38.160370 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160347    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-device-dir\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.160481 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160375    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-sys-fs\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.160481 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160396    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-run\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.160481 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160413    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5972f608-8acf-4650-8b1e-5ad1d2d49bed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.160481 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160444    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-run-netns\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.160481 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160467    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-run-systemd\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.160711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160521    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-run-ovn-kubernetes\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.160711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160543    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-run\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.160711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160577    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a0f84c2-b0b6-4df9-ae88-8b98b4788961-agent-certs\") pod \"konnectivity-agent-hn766\" (UID: \"2a0f84c2-b0b6-4df9-ae88-8b98b4788961\") " pod="kube-system/konnectivity-agent-hn766"
Jul 30 12:59:38.160711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160597    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9rzq\" (UniqueName: \"kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq\") pod \"network-check-target-769q2\" (UID: \"5f461854-f3bc-4026-8c6b-f2b6c756dd17\") " pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:38.160711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160628    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-run-netns\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.160711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160644    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-sysconfig\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.160711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160659    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-var-lib-openvswitch\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.160711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160674    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/573abe38-74b9-49ae-94a6-81dcd54df7ac-ovnkube-script-lib\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.160711 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160691    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcg82\" (UniqueName: \"kubernetes.io/projected/9805b510-a97b-432c-9219-51bea9f4f08f-kube-api-access-gcg82\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.161053 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160722    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-run-netns\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.161053 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160733    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/573abe38-74b9-49ae-94a6-81dcd54df7ac-env-overrides\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.161053 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160762    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-sysconfig\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.161053 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160775    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ae0ce23-6304-4f04-b3a4-c7e3497aa75b-host-slash\") pod \"iptables-alerter-fczjl\" (UID: \"6ae0ce23-6304-4f04-b3a4-c7e3497aa75b\") " pod="openshift-network-operator/iptables-alerter-fczjl"
Jul 30 12:59:38.161053 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160874    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-slash\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.161053 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160910    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ae0ce23-6304-4f04-b3a4-c7e3497aa75b-host-slash\") pod \"iptables-alerter-fczjl\" (UID: \"6ae0ce23-6304-4f04-b3a4-c7e3497aa75b\") " pod="openshift-network-operator/iptables-alerter-fczjl"
Jul 30 12:59:38.161053 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160957    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lwmw\" (UniqueName: \"kubernetes.io/projected/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-kube-api-access-6lwmw\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.161053 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.160985    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-kubelet\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.161053 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161008    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-cni-netd\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.161053 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161034    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9805b510-a97b-432c-9219-51bea9f4f08f-multus-daemon-config\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.161053 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161054    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8t5f\" (UniqueName: \"kubernetes.io/projected/6ae0ce23-6304-4f04-b3a4-c7e3497aa75b-kube-api-access-t8t5f\") pod \"iptables-alerter-fczjl\" (UID: \"6ae0ce23-6304-4f04-b3a4-c7e3497aa75b\") " pod="openshift-network-operator/iptables-alerter-fczjl"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161071    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfg7c\" (UniqueName: \"kubernetes.io/projected/bad62a71-0b34-453b-97c5-bb6ef94cabda-kube-api-access-mfg7c\") pod \"node-resolver-fng7k\" (UID: \"bad62a71-0b34-453b-97c5-bb6ef94cabda\") " pod="openshift-dns/node-resolver-fng7k"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161089    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-systemd-units\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161136    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a0f84c2-b0b6-4df9-ae88-8b98b4788961-konnectivity-ca\") pod \"konnectivity-agent-hn766\" (UID: \"2a0f84c2-b0b6-4df9-ae88-8b98b4788961\") " pod="kube-system/konnectivity-agent-hn766"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161178    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-etc-selinux\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161209    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26774edd-7404-4111-a22e-da6b38958c85-dbus\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161240    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-modprobe-d\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161268    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-var-lib-kubelet\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161295    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/85235635-3099-4830-a013-38009fe3b04f-etc-tuned\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161322    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5972f608-8acf-4650-8b1e-5ad1d2d49bed-cni-binary-copy\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161351    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-multus-cni-dir\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161368    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-var-lib-kubelet\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161371    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-modprobe-d\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161375    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-os-release\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161432    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9805b510-a97b-432c-9219-51bea9f4f08f-cni-binary-copy\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161459    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bad62a71-0b34-453b-97c5-bb6ef94cabda-tmp-dir\") pod \"node-resolver-fng7k\" (UID: \"bad62a71-0b34-453b-97c5-bb6ef94cabda\") " pod="openshift-dns/node-resolver-fng7k"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161487    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-sysctl-d\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.161487 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161474    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-os-release\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.162285 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161516    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5972f608-8acf-4650-8b1e-5ad1d2d49bed-os-release\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.162285 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161527    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5972f608-8acf-4650-8b1e-5ad1d2d49bed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.162285 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161544    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-run-ovn\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.162285 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161570    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-run-multus-certs\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.162285 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161594    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-lib-modules\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.162285 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161652    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5972f608-8acf-4650-8b1e-5ad1d2d49bed-cnibin\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.162285 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161682    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:38.162285 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161704    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-multus-cni-dir\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.162285 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161712    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-etc-openvswitch\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.162285 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161738    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-log-socket\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.162285 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161732    2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 12:59:38.162285 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161747    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9805b510-a97b-432c-9219-51bea9f4f08f-multus-daemon-config\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.162285 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.161849    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5972f608-8acf-4650-8b1e-5ad1d2d49bed-os-release\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.162285 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162250    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-run-multus-certs\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.162949 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162301    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5972f608-8acf-4650-8b1e-5ad1d2d49bed-cni-binary-copy\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.162949 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162367    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-lib-modules\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.162949 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162491    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-cni-bin\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.162949 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162555    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bad62a71-0b34-453b-97c5-bb6ef94cabda-tmp-dir\") pod \"node-resolver-fng7k\" (UID: \"bad62a71-0b34-453b-97c5-bb6ef94cabda\") " pod="openshift-dns/node-resolver-fng7k"
Jul 30 12:59:38.162949 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162566    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e9b91626-80d1-487b-bdf6-75b892b2dd6b-serviceca\") pod \"node-ca-5h277\" (UID: \"e9b91626-80d1-487b-bdf6-75b892b2dd6b\") " pod="openshift-image-registry/node-ca-5h277"
Jul 30 12:59:38.162949 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162637    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9805b510-a97b-432c-9219-51bea9f4f08f-cni-binary-copy\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.162949 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162663    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/573abe38-74b9-49ae-94a6-81dcd54df7ac-ovnkube-config\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.162949 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162733    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/573abe38-74b9-49ae-94a6-81dcd54df7ac-ovn-node-metrics-cert\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.162949 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162774    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd5bj\" (UniqueName: \"kubernetes.io/projected/573abe38-74b9-49ae-94a6-81dcd54df7ac-kube-api-access-zd5bj\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.162949 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162809    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-var-lib-cni-bin\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.162949 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162839    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-multus-conf-dir\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.162949 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162889    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-etc-kubernetes\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.162949 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162944    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-etc-kubernetes\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.163553 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162968    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-var-lib-cni-bin\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.163553 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.162981    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bad62a71-0b34-453b-97c5-bb6ef94cabda-hosts-file\") pod \"node-resolver-fng7k\" (UID: \"bad62a71-0b34-453b-97c5-bb6ef94cabda\") " pod="openshift-dns/node-resolver-fng7k"
Jul 30 12:59:38.163553 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163006    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-multus-conf-dir\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.163553 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163035    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5972f608-8acf-4650-8b1e-5ad1d2d49bed-cnibin\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.163553 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163059    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9b91626-80d1-487b-bdf6-75b892b2dd6b-host\") pod \"node-ca-5h277\" (UID: \"e9b91626-80d1-487b-bdf6-75b892b2dd6b\") " pod="openshift-image-registry/node-ca-5h277"
Jul 30 12:59:38.163553 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163096    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64l85\" (UniqueName: \"kubernetes.io/projected/e9b91626-80d1-487b-bdf6-75b892b2dd6b-kube-api-access-64l85\") pod \"node-ca-5h277\" (UID: \"e9b91626-80d1-487b-bdf6-75b892b2dd6b\") " pod="openshift-image-registry/node-ca-5h277"
Jul 30 12:59:38.163553 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163120    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e9b91626-80d1-487b-bdf6-75b892b2dd6b-serviceca\") pod \"node-ca-5h277\" (UID: \"e9b91626-80d1-487b-bdf6-75b892b2dd6b\") " pod="openshift-image-registry/node-ca-5h277"
Jul 30 12:59:38.163553 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163145    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-sysctl-d\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.163553 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163201    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85235635-3099-4830-a013-38009fe3b04f-tmp\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.163553 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163234    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-var-lib-cni-multus\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.163553 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163500    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bad62a71-0b34-453b-97c5-bb6ef94cabda-hosts-file\") pod \"node-resolver-fng7k\" (UID: \"bad62a71-0b34-453b-97c5-bb6ef94cabda\") " pod="openshift-dns/node-resolver-fng7k"
Jul 30 12:59:38.163553 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163548    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9b91626-80d1-487b-bdf6-75b892b2dd6b-host\") pod \"node-ca-5h277\" (UID: \"e9b91626-80d1-487b-bdf6-75b892b2dd6b\") " pod="openshift-image-registry/node-ca-5h277"
Jul 30 12:59:38.164153 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163582    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-var-lib-kubelet\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.164153 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163620    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-hostroot\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.164153 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163695    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-sysctl-conf\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.164153 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163732    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-systemd\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.164153 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163770    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5972f608-8acf-4650-8b1e-5ad1d2d49bed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.164153 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163830    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-var-lib-cni-multus\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.164153 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163905    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-run-openvswitch\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.164153 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163930    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-var-lib-kubelet\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.164153 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.163934    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-node-log\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.164153 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164017    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5tvx\" (UniqueName: \"kubernetes.io/projected/5972f608-8acf-4650-8b1e-5ad1d2d49bed-kube-api-access-h5tvx\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.164153 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164050    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-sysctl-conf\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.164153 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164089    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.164153 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164104    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-hostroot\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.164153 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164131    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-system-cni-dir\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.164821 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164183    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-systemd\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.164821 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164267    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-run-k8s-cni-cncf-io\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.164821 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164328    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs\") pod \"network-metrics-daemon-74gmj\" (UID: \"c083abff-9168-456b-ba24-dd906dc138b2\") " pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:38.164821 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164393    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.164821 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164435    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26774edd-7404-4111-a22e-da6b38958c85-kubelet-config\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:38.164821 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164529    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-cnibin\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.164821 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164559    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-multus-socket-dir-parent\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.164821 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164603    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6ae0ce23-6304-4f04-b3a4-c7e3497aa75b-iptables-alerter-script\") pod \"iptables-alerter-fczjl\" (UID: \"6ae0ce23-6304-4f04-b3a4-c7e3497aa75b\") " pod="openshift-network-operator/iptables-alerter-fczjl"
Jul 30 12:59:38.164821 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164634    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-kubernetes\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.164821 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164661    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-sys\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.164821 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164714    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-host\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.164821 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164794    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5972f608-8acf-4650-8b1e-5ad1d2d49bed-system-cni-dir\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.164821 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164807    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-cnibin\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.164821 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164823    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7nm8\" (UniqueName: \"kubernetes.io/projected/c083abff-9168-456b-ba24-dd906dc138b2-kube-api-access-d7nm8\") pod \"network-metrics-daemon-74gmj\" (UID: \"c083abff-9168-456b-ba24-dd906dc138b2\") " pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:38.165495 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164842    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcwv5\" (UniqueName: \"kubernetes.io/projected/85235635-3099-4830-a013-38009fe3b04f-kube-api-access-fcwv5\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.165495 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164852    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-multus-socket-dir-parent\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.165495 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164864    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5972f608-8acf-4650-8b1e-5ad1d2d49bed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.165495 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.164890    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-socket-dir\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.165495 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.165108    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-host-run-k8s-cni-cncf-io\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.165495 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.165164    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-host\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.165495 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.165192    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-etc-kubernetes\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.165495 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.165173    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5972f608-8acf-4650-8b1e-5ad1d2d49bed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.165495 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.165225    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85235635-3099-4830-a013-38009fe3b04f-sys\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.165495 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.165268    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-registration-dir\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.165495 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.165352    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6ae0ce23-6304-4f04-b3a4-c7e3497aa75b-iptables-alerter-script\") pod \"iptables-alerter-fczjl\" (UID: \"6ae0ce23-6304-4f04-b3a4-c7e3497aa75b\") " pod="openshift-network-operator/iptables-alerter-fczjl"
Jul 30 12:59:38.165495 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.165370    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9805b510-a97b-432c-9219-51bea9f4f08f-system-cni-dir\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.165495 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.165419    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5972f608-8acf-4650-8b1e-5ad1d2d49bed-system-cni-dir\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.165495 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.165470    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5972f608-8acf-4650-8b1e-5ad1d2d49bed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.166212 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.165542    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 12:59:38.166212 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.165652    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs podName:c083abff-9168-456b-ba24-dd906dc138b2 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:38.665628408 +0000 UTC m=+1.938092809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs") pod "network-metrics-daemon-74gmj" (UID: "c083abff-9168-456b-ba24-dd906dc138b2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 12:59:38.167812 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.167785    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85235635-3099-4830-a013-38009fe3b04f-tmp\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.167905 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.167827    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/85235635-3099-4830-a013-38009fe3b04f-etc-tuned\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.168139 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.168122    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 12:59:38.168222 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.168144    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 12:59:38.168222 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.168158    2557 projected.go:196] Error preparing data for projected volume kube-api-access-r9rzq for pod openshift-network-diagnostics/network-check-target-769q2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 12:59:38.168310 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.168233    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq podName:5f461854-f3bc-4026-8c6b-f2b6c756dd17 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:38.668213795 +0000 UTC m=+1.940678187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r9rzq" (UniqueName: "kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq") pod "network-check-target-769q2" (UID: "5f461854-f3bc-4026-8c6b-f2b6c756dd17") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 12:59:38.171653 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.171623    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcg82\" (UniqueName: \"kubernetes.io/projected/9805b510-a97b-432c-9219-51bea9f4f08f-kube-api-access-gcg82\") pod \"multus-pmvrj\" (UID: \"9805b510-a97b-432c-9219-51bea9f4f08f\") " pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.172877 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.172850    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8t5f\" (UniqueName: \"kubernetes.io/projected/6ae0ce23-6304-4f04-b3a4-c7e3497aa75b-kube-api-access-t8t5f\") pod \"iptables-alerter-fczjl\" (UID: \"6ae0ce23-6304-4f04-b3a4-c7e3497aa75b\") " pod="openshift-network-operator/iptables-alerter-fczjl"
Jul 30 12:59:38.172973 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.172934    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfg7c\" (UniqueName: \"kubernetes.io/projected/bad62a71-0b34-453b-97c5-bb6ef94cabda-kube-api-access-mfg7c\") pod \"node-resolver-fng7k\" (UID: \"bad62a71-0b34-453b-97c5-bb6ef94cabda\") " pod="openshift-dns/node-resolver-fng7k"
Jul 30 12:59:38.173178 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.173150    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64l85\" (UniqueName: \"kubernetes.io/projected/e9b91626-80d1-487b-bdf6-75b892b2dd6b-kube-api-access-64l85\") pod \"node-ca-5h277\" (UID: \"e9b91626-80d1-487b-bdf6-75b892b2dd6b\") " pod="openshift-image-registry/node-ca-5h277"
Jul 30 12:59:38.175059 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.175038    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5tvx\" (UniqueName: \"kubernetes.io/projected/5972f608-8acf-4650-8b1e-5ad1d2d49bed-kube-api-access-h5tvx\") pod \"multus-additional-cni-plugins-7mbgz\" (UID: \"5972f608-8acf-4650-8b1e-5ad1d2d49bed\") " pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.179256 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.179235    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcwv5\" (UniqueName: \"kubernetes.io/projected/85235635-3099-4830-a013-38009fe3b04f-kube-api-access-fcwv5\") pod \"tuned-htmgz\" (UID: \"85235635-3099-4830-a013-38009fe3b04f\") " pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.179350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.179331    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7nm8\" (UniqueName: \"kubernetes.io/projected/c083abff-9168-456b-ba24-dd906dc138b2-kube-api-access-d7nm8\") pod \"network-metrics-daemon-74gmj\" (UID: \"c083abff-9168-456b-ba24-dd906dc138b2\") " pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:38.258861 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:38.258827    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd398727194bd86b778ce466d77f4b02.slice/crio-6040beded45b74c4e376b7874cf956a700b632a352e4832b39bd49e50cbb42d2 WatchSource:0}: Error finding container 6040beded45b74c4e376b7874cf956a700b632a352e4832b39bd49e50cbb42d2: Status 404 returned error can't find the container with id 6040beded45b74c4e376b7874cf956a700b632a352e4832b39bd49e50cbb42d2
Jul 30 12:59:38.266292 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266273    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-socket-dir\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.266398 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266300    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-registration-dir\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.266398 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266316    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-device-dir\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.266398 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266339    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-sys-fs\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.266398 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266366    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-run-netns\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.266398 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266389    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-run-systemd\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.266665 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266414    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-run-ovn-kubernetes\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.266665 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266439    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a0f84c2-b0b6-4df9-ae88-8b98b4788961-agent-certs\") pod \"konnectivity-agent-hn766\" (UID: \"2a0f84c2-b0b6-4df9-ae88-8b98b4788961\") " pod="kube-system/konnectivity-agent-hn766"
Jul 30 12:59:38.266665 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266453    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-socket-dir\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.266665 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266465    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-registration-dir\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.266665 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266479    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-var-lib-openvswitch\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.266665 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266467    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-device-dir\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.266665 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266507    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/573abe38-74b9-49ae-94a6-81dcd54df7ac-ovnkube-script-lib\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.266665 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266570    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/573abe38-74b9-49ae-94a6-81dcd54df7ac-env-overrides\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.266665 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266533    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-var-lib-openvswitch\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.266665 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266652    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-run-netns\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.266665 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266604    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-slash\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.266665 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266683    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266695    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-run-systemd\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266697    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lwmw\" (UniqueName: \"kubernetes.io/projected/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-kube-api-access-6lwmw\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266762    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-sys-fs\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266797    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-slash\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266834    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-run-ovn-kubernetes\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266864    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-kubelet\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266901    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-cni-netd\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266939    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-kubelet\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266956    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-systemd-units\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266970    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-cni-netd\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.266999    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a0f84c2-b0b6-4df9-ae88-8b98b4788961-konnectivity-ca\") pod \"konnectivity-agent-hn766\" (UID: \"2a0f84c2-b0b6-4df9-ae88-8b98b4788961\") " pod="kube-system/konnectivity-agent-hn766"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267013    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-systemd-units\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267026    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-etc-selinux\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267049    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/573abe38-74b9-49ae-94a6-81dcd54df7ac-env-overrides\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267052    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26774edd-7404-4111-a22e-da6b38958c85-dbus\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267100    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26774edd-7404-4111-a22e-da6b38958c85-dbus\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:38.267350 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267116    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-run-ovn\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267133    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/573abe38-74b9-49ae-94a6-81dcd54df7ac-ovnkube-script-lib\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267150    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267179    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-etc-openvswitch\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267188    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-run-ovn\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267190    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-etc-selinux\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267205    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-log-socket\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267231    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-cni-bin\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267252    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-etc-openvswitch\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267260    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/573abe38-74b9-49ae-94a6-81dcd54df7ac-ovnkube-config\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267285    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/573abe38-74b9-49ae-94a6-81dcd54df7ac-ovn-node-metrics-cert\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.267292    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267307    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-log-socket\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267323    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-cni-bin\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.267343    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret podName:26774edd-7404-4111-a22e-da6b38958c85 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:38.767324683 +0000 UTC m=+2.039789062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret") pod "global-pull-secret-syncer-qbp52" (UID: "26774edd-7404-4111-a22e-da6b38958c85") : object "kube-system"/"original-pull-secret" not registered
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267363    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd5bj\" (UniqueName: \"kubernetes.io/projected/573abe38-74b9-49ae-94a6-81dcd54df7ac-kube-api-access-zd5bj\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267402    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-run-openvswitch\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268139 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267429    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-node-log\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268795 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267456    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.268795 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267501    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268795 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267524    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-run-openvswitch\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268795 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267552    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26774edd-7404-4111-a22e-da6b38958c85-kubelet-config\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:38.268795 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267595    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a0f84c2-b0b6-4df9-ae88-8b98b4788961-konnectivity-ca\") pod \"konnectivity-agent-hn766\" (UID: \"2a0f84c2-b0b6-4df9-ae88-8b98b4788961\") " pod="kube-system/konnectivity-agent-hn766"
Jul 30 12:59:38.268795 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267636    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26774edd-7404-4111-a22e-da6b38958c85-kubelet-config\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:38.268795 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267694    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.268795 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267706    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268795 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267747    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/573abe38-74b9-49ae-94a6-81dcd54df7ac-node-log\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.268795 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.267834    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/573abe38-74b9-49ae-94a6-81dcd54df7ac-ovnkube-config\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.269659 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.269636    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a0f84c2-b0b6-4df9-ae88-8b98b4788961-agent-certs\") pod \"konnectivity-agent-hn766\" (UID: \"2a0f84c2-b0b6-4df9-ae88-8b98b4788961\") " pod="kube-system/konnectivity-agent-hn766"
Jul 30 12:59:38.269762 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.269703    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/573abe38-74b9-49ae-94a6-81dcd54df7ac-ovn-node-metrics-cert\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.273037 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:38.273010    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5239ea0e976e2f1f2ffc2d6a67116fe9.slice/crio-d4bc5f20d9994450be1f5ea1e81f8cc92a1b92e1bf5d8b3665a5f1edc6a589d7 WatchSource:0}: Error finding container d4bc5f20d9994450be1f5ea1e81f8cc92a1b92e1bf5d8b3665a5f1edc6a589d7: Status 404 returned error can't find the container with id d4bc5f20d9994450be1f5ea1e81f8cc92a1b92e1bf5d8b3665a5f1edc6a589d7
Jul 30 12:59:38.276396 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.276368    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lwmw\" (UniqueName: \"kubernetes.io/projected/67fdf6a6-01cf-49e5-a9ce-ddab109e73f1-kube-api-access-6lwmw\") pod \"aws-ebs-csi-driver-node-5hkwz\" (UID: \"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.276396 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.276376    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd5bj\" (UniqueName: \"kubernetes.io/projected/573abe38-74b9-49ae-94a6-81dcd54df7ac-kube-api-access-zd5bj\") pod \"ovnkube-node-4q7bg\" (UID: \"573abe38-74b9-49ae-94a6-81dcd54df7ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.377325 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.377237    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-htmgz"
Jul 30 12:59:38.387260 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:38.387234    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85235635_3099_4830_a013_38009fe3b04f.slice/crio-6c3adf4c15f0b2e4a6356325f8241bbcc0e3355eb9e3e4f5f3ce1e905aecbf9d WatchSource:0}: Error finding container 6c3adf4c15f0b2e4a6356325f8241bbcc0e3355eb9e3e4f5f3ce1e905aecbf9d: Status 404 returned error can't find the container with id 6c3adf4c15f0b2e4a6356325f8241bbcc0e3355eb9e3e4f5f3ce1e905aecbf9d
Jul 30 12:59:38.392264 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.392245    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fng7k"
Jul 30 12:59:38.401054 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:38.401032    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad62a71_0b34_453b_97c5_bb6ef94cabda.slice/crio-8a6760768fc4695494c871f58c025a523360f7662dc2e6fd722363214df85624 WatchSource:0}: Error finding container 8a6760768fc4695494c871f58c025a523360f7662dc2e6fd722363214df85624: Status 404 returned error can't find the container with id 8a6760768fc4695494c871f58c025a523360f7662dc2e6fd722363214df85624
Jul 30 12:59:38.416911 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.416888    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5h277"
Jul 30 12:59:38.425364 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:38.425340    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9b91626_80d1_487b_bdf6_75b892b2dd6b.slice/crio-c6f563bbaac041144768c201d979c93c34653bd2ce4f972f8ef26d122304a144 WatchSource:0}: Error finding container c6f563bbaac041144768c201d979c93c34653bd2ce4f972f8ef26d122304a144: Status 404 returned error can't find the container with id c6f563bbaac041144768c201d979c93c34653bd2ce4f972f8ef26d122304a144
Jul 30 12:59:38.433525 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.433507    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7mbgz"
Jul 30 12:59:38.441887 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:38.441862    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5972f608_8acf_4650_8b1e_5ad1d2d49bed.slice/crio-4673b5ff4760b2730d03f924987b2942a235045781ea7282b1a4ffbedb5a0553 WatchSource:0}: Error finding container 4673b5ff4760b2730d03f924987b2942a235045781ea7282b1a4ffbedb5a0553: Status 404 returned error can't find the container with id 4673b5ff4760b2730d03f924987b2942a235045781ea7282b1a4ffbedb5a0553
Jul 30 12:59:38.447768 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.447752    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pmvrj"
Jul 30 12:59:38.454802 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.454782    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fczjl"
Jul 30 12:59:38.455596 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:38.455577    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9805b510_a97b_432c_9219_51bea9f4f08f.slice/crio-5b9d06f1f9c2f5b8f2e1ad812cb3c31355177b4167c8811f2e833a52953f0530 WatchSource:0}: Error finding container 5b9d06f1f9c2f5b8f2e1ad812cb3c31355177b4167c8811f2e833a52953f0530: Status 404 returned error can't find the container with id 5b9d06f1f9c2f5b8f2e1ad812cb3c31355177b4167c8811f2e833a52953f0530
Jul 30 12:59:38.463786 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:38.463764    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae0ce23_6304_4f04_b3a4_c7e3497aa75b.slice/crio-0bb6b07c573bc0543a4a46998dd7a8789c635c694e3d965a526301a3d7862f82 WatchSource:0}: Error finding container 0bb6b07c573bc0543a4a46998dd7a8789c635c694e3d965a526301a3d7862f82: Status 404 returned error can't find the container with id 0bb6b07c573bc0543a4a46998dd7a8789c635c694e3d965a526301a3d7862f82
Jul 30 12:59:38.468376 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.468355    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 12:59:38.476796 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:38.476774    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod573abe38_74b9_49ae_94a6_81dcd54df7ac.slice/crio-9d2b04d4351f5b2e49b495c51e054c00a8239a42c66c96a0eea76a1e03cba4cf WatchSource:0}: Error finding container 9d2b04d4351f5b2e49b495c51e054c00a8239a42c66c96a0eea76a1e03cba4cf: Status 404 returned error can't find the container with id 9d2b04d4351f5b2e49b495c51e054c00a8239a42c66c96a0eea76a1e03cba4cf
Jul 30 12:59:38.494318 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.494296    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hn766"
Jul 30 12:59:38.499090 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.499072    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz"
Jul 30 12:59:38.503830 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:38.503804    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a0f84c2_b0b6_4df9_ae88_8b98b4788961.slice/crio-702fe560264f8eabf436dec2bcd984dd18dea040ce3cbce44ec131dad001c8d3 WatchSource:0}: Error finding container 702fe560264f8eabf436dec2bcd984dd18dea040ce3cbce44ec131dad001c8d3: Status 404 returned error can't find the container with id 702fe560264f8eabf436dec2bcd984dd18dea040ce3cbce44ec131dad001c8d3
Jul 30 12:59:38.507957 ip-10-0-143-33 kubenswrapper[2557]: W0730 12:59:38.507935    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67fdf6a6_01cf_49e5_a9ce_ddab109e73f1.slice/crio-500a361ec799b680b7ce8cc9ffdc711ae8dc54a055e6af4481722c5fef1a4d55 WatchSource:0}: Error finding container 500a361ec799b680b7ce8cc9ffdc711ae8dc54a055e6af4481722c5fef1a4d55: Status 404 returned error can't find the container with id 500a361ec799b680b7ce8cc9ffdc711ae8dc54a055e6af4481722c5fef1a4d55
Jul 30 12:59:38.670921 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.670824    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs\") pod \"network-metrics-daemon-74gmj\" (UID: \"c083abff-9168-456b-ba24-dd906dc138b2\") " pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:38.670921 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.670880    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9rzq\" (UniqueName: \"kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq\") pod \"network-check-target-769q2\" (UID: \"5f461854-f3bc-4026-8c6b-f2b6c756dd17\") " pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:38.671113 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.670990    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 12:59:38.671113 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.671013    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 12:59:38.671113 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.671031    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 12:59:38.671113 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.671045    2557 projected.go:196] Error preparing data for projected volume kube-api-access-r9rzq for pod openshift-network-diagnostics/network-check-target-769q2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 12:59:38.671113 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.671060    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs podName:c083abff-9168-456b-ba24-dd906dc138b2 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:39.671040472 +0000 UTC m=+2.943504862 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs") pod "network-metrics-daemon-74gmj" (UID: "c083abff-9168-456b-ba24-dd906dc138b2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 12:59:38.671113 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.671082    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq podName:5f461854-f3bc-4026-8c6b-f2b6c756dd17 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:39.671071119 +0000 UTC m=+2.943535495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r9rzq" (UniqueName: "kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq") pod "network-check-target-769q2" (UID: "5f461854-f3bc-4026-8c6b-f2b6c756dd17") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 12:59:38.772352 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.772317    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:38.772520 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.772484    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 12:59:38.772587 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:38.772556    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret podName:26774edd-7404-4111-a22e-da6b38958c85 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:39.772539397 +0000 UTC m=+3.045003776 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret") pod "global-pull-secret-syncer-qbp52" (UID: "26774edd-7404-4111-a22e-da6b38958c85") : object "kube-system"/"original-pull-secret" not registered
Jul 30 12:59:38.926282 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:38.926201    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 12:59:39.105899 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.105821    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 12:54:38 +0000 UTC" deadline="2028-01-27 22:51:31.568345859 +0000 UTC"
Jul 30 12:59:39.105899 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.105897    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13113h51m52.46245461s"
Jul 30 12:59:39.177873 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.177799    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:39.178046 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:39.177931    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 12:59:39.194727 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.194667    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5h277" event={"ID":"e9b91626-80d1-487b-bdf6-75b892b2dd6b","Type":"ContainerStarted","Data":"c6f563bbaac041144768c201d979c93c34653bd2ce4f972f8ef26d122304a144"}
Jul 30 12:59:39.205783 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.205698    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fng7k" event={"ID":"bad62a71-0b34-453b-97c5-bb6ef94cabda","Type":"ContainerStarted","Data":"8a6760768fc4695494c871f58c025a523360f7662dc2e6fd722363214df85624"}
Jul 30 12:59:39.208756 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.208598    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz" event={"ID":"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1","Type":"ContainerStarted","Data":"500a361ec799b680b7ce8cc9ffdc711ae8dc54a055e6af4481722c5fef1a4d55"}
Jul 30 12:59:39.217804 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.217775    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hn766" event={"ID":"2a0f84c2-b0b6-4df9-ae88-8b98b4788961","Type":"ContainerStarted","Data":"702fe560264f8eabf436dec2bcd984dd18dea040ce3cbce44ec131dad001c8d3"}
Jul 30 12:59:39.224564 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.224518    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pmvrj" event={"ID":"9805b510-a97b-432c-9219-51bea9f4f08f","Type":"ContainerStarted","Data":"5b9d06f1f9c2f5b8f2e1ad812cb3c31355177b4167c8811f2e833a52953f0530"}
Jul 30 12:59:39.226911 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.226883    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7mbgz" event={"ID":"5972f608-8acf-4650-8b1e-5ad1d2d49bed","Type":"ContainerStarted","Data":"4673b5ff4760b2730d03f924987b2942a235045781ea7282b1a4ffbedb5a0553"}
Jul 30 12:59:39.229197 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.229173    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-htmgz" event={"ID":"85235635-3099-4830-a013-38009fe3b04f","Type":"ContainerStarted","Data":"6c3adf4c15f0b2e4a6356325f8241bbcc0e3355eb9e3e4f5f3ce1e905aecbf9d"}
Jul 30 12:59:39.237629 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.237600    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal" event={"ID":"5239ea0e976e2f1f2ffc2d6a67116fe9","Type":"ContainerStarted","Data":"d4bc5f20d9994450be1f5ea1e81f8cc92a1b92e1bf5d8b3665a5f1edc6a589d7"}
Jul 30 12:59:39.259223 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.259176    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-33.ec2.internal" event={"ID":"bd398727194bd86b778ce466d77f4b02","Type":"ContainerStarted","Data":"6040beded45b74c4e376b7874cf956a700b632a352e4832b39bd49e50cbb42d2"}
Jul 30 12:59:39.274959 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.274920    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg" event={"ID":"573abe38-74b9-49ae-94a6-81dcd54df7ac","Type":"ContainerStarted","Data":"9d2b04d4351f5b2e49b495c51e054c00a8239a42c66c96a0eea76a1e03cba4cf"}
Jul 30 12:59:39.281389 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.281346    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fczjl" event={"ID":"6ae0ce23-6304-4f04-b3a4-c7e3497aa75b","Type":"ContainerStarted","Data":"0bb6b07c573bc0543a4a46998dd7a8789c635c694e3d965a526301a3d7862f82"}
Jul 30 12:59:39.478994 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.478905    2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 12:59:39.680647 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.680597    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs\") pod \"network-metrics-daemon-74gmj\" (UID: \"c083abff-9168-456b-ba24-dd906dc138b2\") " pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:39.680853 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.680667    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9rzq\" (UniqueName: \"kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq\") pod \"network-check-target-769q2\" (UID: \"5f461854-f3bc-4026-8c6b-f2b6c756dd17\") " pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:39.680853 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:39.680816    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 12:59:39.680853 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:39.680837    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 12:59:39.680853 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:39.680850    2557 projected.go:196] Error preparing data for projected volume kube-api-access-r9rzq for pod openshift-network-diagnostics/network-check-target-769q2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 12:59:39.681104 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:39.680908    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq podName:5f461854-f3bc-4026-8c6b-f2b6c756dd17 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:41.680889312 +0000 UTC m=+4.953353694 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r9rzq" (UniqueName: "kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq") pod "network-check-target-769q2" (UID: "5f461854-f3bc-4026-8c6b-f2b6c756dd17") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 12:59:39.681104 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:39.681002    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 12:59:39.681104 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:39.681044    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs podName:c083abff-9168-456b-ba24-dd906dc138b2 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:41.681029923 +0000 UTC m=+4.953494303 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs") pod "network-metrics-daemon-74gmj" (UID: "c083abff-9168-456b-ba24-dd906dc138b2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 12:59:39.781678 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:39.781584    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:39.781864 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:39.781811    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 12:59:39.781936 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:39.781891    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret podName:26774edd-7404-4111-a22e-da6b38958c85 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:41.781867993 +0000 UTC m=+5.054332373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret") pod "global-pull-secret-syncer-qbp52" (UID: "26774edd-7404-4111-a22e-da6b38958c85") : object "kube-system"/"original-pull-secret" not registered
Jul 30 12:59:40.106214 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:40.106096    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 12:54:38 +0000 UTC" deadline="2028-02-06 17:16:59.410901159 +0000 UTC"
Jul 30 12:59:40.106214 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:40.106137    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13348h17m19.304770261s"
Jul 30 12:59:40.177427 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:40.177397    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:40.177652 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:40.177539    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 12:59:40.178021 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:40.177999    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:40.178149 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:40.178113    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 12:59:41.200370 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:41.200341    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:41.200890 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:41.200485    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 12:59:41.701543 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:41.701503    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs\") pod \"network-metrics-daemon-74gmj\" (UID: \"c083abff-9168-456b-ba24-dd906dc138b2\") " pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:41.701741 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:41.701573    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9rzq\" (UniqueName: \"kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq\") pod \"network-check-target-769q2\" (UID: \"5f461854-f3bc-4026-8c6b-f2b6c756dd17\") " pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:41.701829 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:41.701758    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 12:59:41.701829 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:41.701778    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 12:59:41.701829 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:41.701793    2557 projected.go:196] Error preparing data for projected volume kube-api-access-r9rzq for pod openshift-network-diagnostics/network-check-target-769q2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 12:59:41.701983 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:41.701854    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq podName:5f461854-f3bc-4026-8c6b-f2b6c756dd17 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:45.701835243 +0000 UTC m=+8.974299625 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r9rzq" (UniqueName: "kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq") pod "network-check-target-769q2" (UID: "5f461854-f3bc-4026-8c6b-f2b6c756dd17") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 12:59:41.702546 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:41.702344    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 12:59:41.702546 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:41.702404    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs podName:c083abff-9168-456b-ba24-dd906dc138b2 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:45.702386866 +0000 UTC m=+8.974851248 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs") pod "network-metrics-daemon-74gmj" (UID: "c083abff-9168-456b-ba24-dd906dc138b2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 12:59:41.802767 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:41.802712    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:41.802974 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:41.802950    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 12:59:41.803102 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:41.803087    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret podName:26774edd-7404-4111-a22e-da6b38958c85 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:45.803065018 +0000 UTC m=+9.075529412 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret") pod "global-pull-secret-syncer-qbp52" (UID: "26774edd-7404-4111-a22e-da6b38958c85") : object "kube-system"/"original-pull-secret" not registered
Jul 30 12:59:42.177335 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:42.177300    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:42.177521 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:42.177426    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 12:59:42.177847 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:42.177803    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:42.177943 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:42.177898    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 12:59:43.180430 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:43.178080    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:43.180430 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:43.178215    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 12:59:44.177830 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:44.177790    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:44.177830 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:44.177835    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:44.178142 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:44.177952    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 12:59:44.178142 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:44.178098    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 12:59:45.181681 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:45.181636    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:45.182135 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:45.181776    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 12:59:45.737504 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:45.737463    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9rzq\" (UniqueName: \"kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq\") pod \"network-check-target-769q2\" (UID: \"5f461854-f3bc-4026-8c6b-f2b6c756dd17\") " pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:45.737734 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:45.737568    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs\") pod \"network-metrics-daemon-74gmj\" (UID: \"c083abff-9168-456b-ba24-dd906dc138b2\") " pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:45.737734 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:45.737694    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 12:59:45.737886 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:45.737768    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs podName:c083abff-9168-456b-ba24-dd906dc138b2 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:53.737750065 +0000 UTC m=+17.010214463 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs") pod "network-metrics-daemon-74gmj" (UID: "c083abff-9168-456b-ba24-dd906dc138b2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 12:59:45.738109 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:45.737698    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 12:59:45.738259 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:45.738123    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 12:59:45.738259 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:45.738140    2557 projected.go:196] Error preparing data for projected volume kube-api-access-r9rzq for pod openshift-network-diagnostics/network-check-target-769q2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 12:59:45.738259 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:45.738197    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq podName:5f461854-f3bc-4026-8c6b-f2b6c756dd17 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:53.738179818 +0000 UTC m=+17.010644214 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r9rzq" (UniqueName: "kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq") pod "network-check-target-769q2" (UID: "5f461854-f3bc-4026-8c6b-f2b6c756dd17") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 12:59:45.839194 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:45.838539    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:45.839194 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:45.838777    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 12:59:45.839194 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:45.838841    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret podName:26774edd-7404-4111-a22e-da6b38958c85 nodeName:}" failed. No retries permitted until 2026-07-30 12:59:53.838823333 +0000 UTC m=+17.111287714 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret") pod "global-pull-secret-syncer-qbp52" (UID: "26774edd-7404-4111-a22e-da6b38958c85") : object "kube-system"/"original-pull-secret" not registered
Jul 30 12:59:46.178225 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:46.177675    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:46.178225 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:46.177829    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 12:59:46.178560 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:46.178534    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:46.178712 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:46.178696    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 12:59:47.179793 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:47.178598    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:47.179793 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:47.178737    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 12:59:48.178272 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:48.178035    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:48.178272 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:48.178190    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 12:59:48.178272 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:48.178035    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:48.178526 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:48.178292    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 12:59:49.180474 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:49.180441    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:49.180872 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:49.180546    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 12:59:50.178248 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:50.178214    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:50.178248 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:50.178249    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:50.178488 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:50.178349    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 12:59:50.178546 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:50.178481    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 12:59:51.177470 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:51.177435    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:51.177912 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:51.177554    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 12:59:52.178369 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:52.178333    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:52.178369 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:52.178363    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:52.178928 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:52.178478    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 12:59:52.178928 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:52.178578    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 12:59:53.178310 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:53.178281    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:53.178487 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:53.178390    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 12:59:53.799429 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:53.799393    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs\") pod \"network-metrics-daemon-74gmj\" (UID: \"c083abff-9168-456b-ba24-dd906dc138b2\") " pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:53.799615 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:53.799441    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9rzq\" (UniqueName: \"kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq\") pod \"network-check-target-769q2\" (UID: \"5f461854-f3bc-4026-8c6b-f2b6c756dd17\") " pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:53.799615 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:53.799563    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 12:59:53.799615 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:53.799564    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 12:59:53.799769 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:53.799622    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 12:59:53.799769 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:53.799632    2557 projected.go:196] Error preparing data for projected volume kube-api-access-r9rzq for pod openshift-network-diagnostics/network-check-target-769q2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 12:59:53.799769 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:53.799659    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs podName:c083abff-9168-456b-ba24-dd906dc138b2 nodeName:}" failed. No retries permitted until 2026-07-30 13:00:09.799636444 +0000 UTC m=+33.072100822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs") pod "network-metrics-daemon-74gmj" (UID: "c083abff-9168-456b-ba24-dd906dc138b2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 12:59:53.799769 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:53.799690    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq podName:5f461854-f3bc-4026-8c6b-f2b6c756dd17 nodeName:}" failed. No retries permitted until 2026-07-30 13:00:09.799668541 +0000 UTC m=+33.072132921 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r9rzq" (UniqueName: "kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq") pod "network-check-target-769q2" (UID: "5f461854-f3bc-4026-8c6b-f2b6c756dd17") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 12:59:53.900658 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:53.900614    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:53.900873 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:53.900786    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 12:59:53.900945 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:53.900871    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret podName:26774edd-7404-4111-a22e-da6b38958c85 nodeName:}" failed. No retries permitted until 2026-07-30 13:00:09.900848706 +0000 UTC m=+33.173313083 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret") pod "global-pull-secret-syncer-qbp52" (UID: "26774edd-7404-4111-a22e-da6b38958c85") : object "kube-system"/"original-pull-secret" not registered
Jul 30 12:59:54.178129 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:54.178097    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:54.178286 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:54.178134    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:54.178286 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:54.178216    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 12:59:54.178359 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:54.178317    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 12:59:55.177800 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:55.177766    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:55.178304 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:55.177887    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 12:59:56.178117 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:56.178082    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:56.178542 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:56.178124    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:56.178542 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:56.178214    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 12:59:56.178542 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:56.178351    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 12:59:57.178116 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:57.178080    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:57.178301 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:57.178191    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 12:59:58.178264 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:58.178224    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 12:59:58.178445 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:58.178362    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 12:59:58.178445 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:58.178422    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 12:59:58.178867 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:58.178529    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 12:59:59.178280 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.177992    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 12:59:59.178435 ip-10-0-143-33 kubenswrapper[2557]: E0730 12:59:59.178381    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 12:59:59.337992 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.337962    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pmvrj" event={"ID":"9805b510-a97b-432c-9219-51bea9f4f08f","Type":"ContainerStarted","Data":"278a9bafdcac018d01091c7a628ef875364294bd655c5fb282aa4076987150b5"}
Jul 30 12:59:59.339524 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.339503    2557 generic.go:356] "Generic (PLEG): container finished" podID="5972f608-8acf-4650-8b1e-5ad1d2d49bed" containerID="4852a8d51a62b6f6aaf450b5c7c79bad46b72e43dd5dc393873823ba70f28fb1" exitCode=0
Jul 30 12:59:59.339626 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.339582    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7mbgz" event={"ID":"5972f608-8acf-4650-8b1e-5ad1d2d49bed","Type":"ContainerDied","Data":"4852a8d51a62b6f6aaf450b5c7c79bad46b72e43dd5dc393873823ba70f28fb1"}
Jul 30 12:59:59.341050 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.341030    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-htmgz" event={"ID":"85235635-3099-4830-a013-38009fe3b04f","Type":"ContainerStarted","Data":"b895459d731634b836a86cdac01df80ba4881d308b66d6fa896c613956bc1d09"}
Jul 30 12:59:59.342674 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.342652    2557 generic.go:356] "Generic (PLEG): container finished" podID="5239ea0e976e2f1f2ffc2d6a67116fe9" containerID="46e3d00a9251f89762df8868659bfbf9ca7a0333f8ef41ff048e718e9389f338" exitCode=0
Jul 30 12:59:59.342759 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.342734    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal" event={"ID":"5239ea0e976e2f1f2ffc2d6a67116fe9","Type":"ContainerDied","Data":"46e3d00a9251f89762df8868659bfbf9ca7a0333f8ef41ff048e718e9389f338"}
Jul 30 12:59:59.344036 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.344012    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-33.ec2.internal" event={"ID":"bd398727194bd86b778ce466d77f4b02","Type":"ContainerStarted","Data":"be8a2de2f7aea59bd8b5374eaf9e2738970e2fe561540707a5dcbc477ba3b558"}
Jul 30 12:59:59.346272 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.346251    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg" event={"ID":"573abe38-74b9-49ae-94a6-81dcd54df7ac","Type":"ContainerStarted","Data":"423d1e7f814251231319c79e8412d800c0b44c12092ae803b8bb7f2abc68f0a4"}
Jul 30 12:59:59.346338 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.346302    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg" event={"ID":"573abe38-74b9-49ae-94a6-81dcd54df7ac","Type":"ContainerStarted","Data":"76dfed199c07b80b06fb4df9719cbf759909a4d377b36a5a072bf1f1b4de8288"}
Jul 30 12:59:59.346381 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.346342    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg" event={"ID":"573abe38-74b9-49ae-94a6-81dcd54df7ac","Type":"ContainerStarted","Data":"8be66376a2a0c023fd9d5a9abe4b820d816c2f4920a5229c1f4f622b3185a172"}
Jul 30 12:59:59.346418 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.346380    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg" event={"ID":"573abe38-74b9-49ae-94a6-81dcd54df7ac","Type":"ContainerStarted","Data":"a04a6f7da67b58a1b4ed11a843f804ce7990fd531cfe8f86338a867409c0e03b"}
Jul 30 12:59:59.347405 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.347384    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5h277" event={"ID":"e9b91626-80d1-487b-bdf6-75b892b2dd6b","Type":"ContainerStarted","Data":"94a9df623ff870b5d671ea4aa327a6c17ce54da7ad6fc4277184af92ee21e220"}
Jul 30 12:59:59.348583 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.348531    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fng7k" event={"ID":"bad62a71-0b34-453b-97c5-bb6ef94cabda","Type":"ContainerStarted","Data":"045322e7eb14d669453db8248d34369350b442d7cda37d94423d2af33377b6d0"}
Jul 30 12:59:59.350048 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.350031    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz" event={"ID":"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1","Type":"ContainerStarted","Data":"5af592c9f5dd1bd359a99eb65f9434823d13aae1f9d8efda8d00d4b068dd1695"}
Jul 30 12:59:59.351221 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.351201    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hn766" event={"ID":"2a0f84c2-b0b6-4df9-ae88-8b98b4788961","Type":"ContainerStarted","Data":"dd55a789077400b88f4ee8ccbb49c95c1aa958ffd03559670488d0605989075d"}
Jul 30 12:59:59.356936 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.356892    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pmvrj" podStartSLOduration=0.95112243 podStartE2EDuration="21.356880731s" totalImagesPullingTime="20.405758301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 12:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 12:59:59.356372892 +0000 UTC m=+22.628837289" watchObservedRunningTime="2026-07-30 12:59:59.356880731 +0000 UTC m=+22.629345128"
Jul 30 12:59:59.372108 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.372072    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fng7k" podStartSLOduration=0.964413931 podStartE2EDuration="21.372058935s" totalImagesPullingTime="20.407645004s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 12:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 12:59:59.371679738 +0000 UTC m=+22.644144149" watchObservedRunningTime="2026-07-30 12:59:59.372058935 +0000 UTC m=+22.644523331"
Jul 30 12:59:59.385696 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.385661    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5h277" podStartSLOduration=1.009575866 podStartE2EDuration="21.385649499s" totalImagesPullingTime="20.376073633s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 12:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 12:59:59.385397803 +0000 UTC m=+22.657862219" watchObservedRunningTime="2026-07-30 12:59:59.385649499 +0000 UTC m=+22.658113896"
Jul 30 12:59:59.437008 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.436968    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-33.ec2.internal" podStartSLOduration=21.436954467 podStartE2EDuration="21.436954467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 12:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 12:59:59.418897562 +0000 UTC m=+22.691361992" watchObservedRunningTime="2026-07-30 12:59:59.436954467 +0000 UTC m=+22.709418913"
Jul 30 12:59:59.464426 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.464267    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-htmgz" podStartSLOduration=1.05654885 podStartE2EDuration="21.464252617s" totalImagesPullingTime="20.407703767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 12:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 12:59:59.436447858 +0000 UTC m=+22.708912255" watchObservedRunningTime="2026-07-30 12:59:59.464252617 +0000 UTC m=+22.736717014"
Jul 30 12:59:59.480321 ip-10-0-143-33 kubenswrapper[2557]: I0730 12:59:59.480286    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hn766" podStartSLOduration=1.189718162 podStartE2EDuration="21.480272158s" totalImagesPullingTime="20.290553996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 12:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 12:59:59.47956063 +0000 UTC m=+22.752025045" watchObservedRunningTime="2026-07-30 12:59:59.480272158 +0000 UTC m=+22.752736555"
Jul 30 13:00:00.177711 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:00.177675    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 13:00:00.177711 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:00.177711    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 13:00:00.177973 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:00.177776    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 13:00:00.177973 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:00.177888    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 13:00:00.355934 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:00.355892    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal" event={"ID":"5239ea0e976e2f1f2ffc2d6a67116fe9","Type":"ContainerStarted","Data":"29e77adeed4ebdb14bd158b94cbee4f9915583e2ff8908dd8eb92673392bed95"}
Jul 30 13:00:00.359498 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:00.359471    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg" event={"ID":"573abe38-74b9-49ae-94a6-81dcd54df7ac","Type":"ContainerStarted","Data":"c12cbd997741c9c5a5e4c3c20abd5152ccb8d6d80dad48d8817be65f7105bf96"}
Jul 30 13:00:00.359633 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:00.359555    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg" event={"ID":"573abe38-74b9-49ae-94a6-81dcd54df7ac","Type":"ContainerStarted","Data":"1cb4345a97eca62b01e8b54f9498c0dde0a52ddd6790571393eff7682089db12"}
Jul 30 13:00:00.361160 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:00.361132    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fczjl" event={"ID":"6ae0ce23-6304-4f04-b3a4-c7e3497aa75b","Type":"ContainerStarted","Data":"7c1470fbcd6e96fd02846e9589850c4443a99238b6deedd80a720a8bc150392b"}
Jul 30 13:00:00.372763 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:00.372715    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-33.ec2.internal" podStartSLOduration=22.372697014 podStartE2EDuration="22.372697014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 12:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:00:00.372635959 +0000 UTC m=+23.645100357" watchObservedRunningTime="2026-07-30 13:00:00.372697014 +0000 UTC m=+23.645161412"
Jul 30 13:00:00.386738 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:00.386693    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fczjl" podStartSLOduration=2.047855364 podStartE2EDuration="22.386679354s" totalImagesPullingTime="20.33882399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 12:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:00:00.386276106 +0000 UTC m=+23.658740528" watchObservedRunningTime="2026-07-30 13:00:00.386679354 +0000 UTC m=+23.659143750"
Jul 30 13:00:00.419383 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:00.419357    2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 13:00:01.120538 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:01.120431    2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T13:00:00.419379288Z","UUID":"3068cc56-7f1d-440f-94d3-50f916e6fd32","Handler":null,"Name":"","Endpoint":""}
Jul 30 13:00:01.122190 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:01.122165    2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 13:00:01.122326 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:01.122207    2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 13:00:01.178312 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:01.178279    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 13:00:01.178468 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:01.178410    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 13:00:01.366025 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:01.365984    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz" event={"ID":"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1","Type":"ContainerStarted","Data":"183209a811dbdb72e6552dc238f9e307f906d3c4b6af831e63651276f6997f52"}
Jul 30 13:00:02.177606 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:02.177576    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 13:00:02.177606 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:02.177584    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 13:00:02.177963 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:02.177708    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 13:00:02.177963 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:02.177873    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 13:00:02.372300 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:02.372260    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg" event={"ID":"573abe38-74b9-49ae-94a6-81dcd54df7ac","Type":"ContainerStarted","Data":"0b3e11053cafaaa9b625ce10ac2b3033b7fb53fb66a9c602a3d6177f415c0b93"}
Jul 30 13:00:02.374449 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:02.374417    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz" event={"ID":"67fdf6a6-01cf-49e5-a9ce-ddab109e73f1","Type":"ContainerStarted","Data":"8cf033bf8097a3acdd8ef71431a84f50b9f1650e89d19a859f21bc4472b11f68"}
Jul 30 13:00:03.061236 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:03.061192    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hn766"
Jul 30 13:00:03.061852 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:03.061827    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hn766"
Jul 30 13:00:03.077955 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:03.077905    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5hkwz" podStartSLOduration=2.33643276 podStartE2EDuration="25.077883433s" totalImagesPullingTime="22.741450673s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 12:59:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:00:02.392747317 +0000 UTC m=+25.665211726" watchObservedRunningTime="2026-07-30 13:00:03.077883433 +0000 UTC m=+26.350347829"
Jul 30 13:00:03.178240 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:03.178206    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 13:00:03.178413 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:03.178330    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 13:00:03.377784 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:03.377639    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hn766"
Jul 30 13:00:03.378207 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:03.378052    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hn766"
Jul 30 13:00:04.177650 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:04.177445    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 13:00:04.177802 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:04.177525    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 13:00:04.177802 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:04.177769    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 13:00:04.177929 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:04.177839    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 13:00:04.381724 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:04.381619    2557 generic.go:356] "Generic (PLEG): container finished" podID="5972f608-8acf-4650-8b1e-5ad1d2d49bed" containerID="82db436b441009942c199216201ad2f7f7bef49a23295f58c632e6ddb616e903" exitCode=0
Jul 30 13:00:04.382329 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:04.381711    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7mbgz" event={"ID":"5972f608-8acf-4650-8b1e-5ad1d2d49bed","Type":"ContainerDied","Data":"82db436b441009942c199216201ad2f7f7bef49a23295f58c632e6ddb616e903"}
Jul 30 13:00:04.385407 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:04.385380    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg" event={"ID":"573abe38-74b9-49ae-94a6-81dcd54df7ac","Type":"ContainerStarted","Data":"e06276d3115f002dcd335192cf17fecb6eff12273cfc96e774b6a61607264e7c"}
Jul 30 13:00:05.177890 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:05.177847    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 13:00:05.178113 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:05.177989    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 13:00:05.389301 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:05.389276    2557 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 30 13:00:05.390056 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:05.389822    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 13:00:05.390056 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:05.389913    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 13:00:05.407488 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:05.407200    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 13:00:05.407999 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:05.407932    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 13:00:05.439794 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:05.439204    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg" podStartSLOduration=7.051207385 podStartE2EDuration="27.439184212s" totalImagesPullingTime="20.387976827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 12:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:00:04.430669272 +0000 UTC m=+27.703133685" watchObservedRunningTime="2026-07-30 13:00:05.439184212 +0000 UTC m=+28.711648603"
Jul 30 13:00:05.778827 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:05.778562    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 13:00:06.177786 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:06.177754    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 13:00:06.177786 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:06.177783    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 13:00:06.178078 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:06.177893    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 13:00:06.178078 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:06.178007    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 13:00:06.250204 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:06.250166    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qbp52"]
Jul 30 13:00:06.253821 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:06.253453    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-74gmj"]
Jul 30 13:00:06.254248 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:06.254223    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-769q2"]
Jul 30 13:00:06.254373 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:06.254340    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 13:00:06.254461 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:06.254437    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 13:00:06.392189 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:06.392107    2557 generic.go:356] "Generic (PLEG): container finished" podID="5972f608-8acf-4650-8b1e-5ad1d2d49bed" containerID="52624211a70b997f5f125a6fcf1bece9a6980cc324465643f749ad03d078f61e" exitCode=0
Jul 30 13:00:06.392189 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:06.392165    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7mbgz" event={"ID":"5972f608-8acf-4650-8b1e-5ad1d2d49bed","Type":"ContainerDied","Data":"52624211a70b997f5f125a6fcf1bece9a6980cc324465643f749ad03d078f61e"}
Jul 30 13:00:06.392189 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:06.392188    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 13:00:06.392757 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:06.392238    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 13:00:06.392757 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:06.392352    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 13:00:06.392757 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:06.392731    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 13:00:08.177840 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:08.177793    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 13:00:08.178222 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:08.177842    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 13:00:08.178222 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:08.177809    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 13:00:08.178222 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:08.177927    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 13:00:08.178222 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:08.177978    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 13:00:08.178222 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:08.178062    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 13:00:08.398136 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:08.398046    2557 generic.go:356] "Generic (PLEG): container finished" podID="5972f608-8acf-4650-8b1e-5ad1d2d49bed" containerID="772c316ee34e46ee177ef67bd1b830dbdd643f0d763f7a96d773bc4d9777f061" exitCode=0
Jul 30 13:00:08.398136 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:08.398113    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7mbgz" event={"ID":"5972f608-8acf-4650-8b1e-5ad1d2d49bed","Type":"ContainerDied","Data":"772c316ee34e46ee177ef67bd1b830dbdd643f0d763f7a96d773bc4d9777f061"}
Jul 30 13:00:09.823437 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:09.823406    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs\") pod \"network-metrics-daemon-74gmj\" (UID: \"c083abff-9168-456b-ba24-dd906dc138b2\") " pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 13:00:09.823867 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:09.823453    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9rzq\" (UniqueName: \"kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq\") pod \"network-check-target-769q2\" (UID: \"5f461854-f3bc-4026-8c6b-f2b6c756dd17\") " pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 13:00:09.823867 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:09.823563    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 13:00:09.823867 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:09.823575    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 13:00:09.823867 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:09.823606    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 13:00:09.823867 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:09.823619    2557 projected.go:196] Error preparing data for projected volume kube-api-access-r9rzq for pod openshift-network-diagnostics/network-check-target-769q2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 13:00:09.823867 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:09.823649    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs podName:c083abff-9168-456b-ba24-dd906dc138b2 nodeName:}" failed. No retries permitted until 2026-07-30 13:00:41.823632525 +0000 UTC m=+65.096096914 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs") pod "network-metrics-daemon-74gmj" (UID: "c083abff-9168-456b-ba24-dd906dc138b2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 13:00:09.823867 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:09.823666    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq podName:5f461854-f3bc-4026-8c6b-f2b6c756dd17 nodeName:}" failed. No retries permitted until 2026-07-30 13:00:41.823659119 +0000 UTC m=+65.096123494 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r9rzq" (UniqueName: "kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq") pod "network-check-target-769q2" (UID: "5f461854-f3bc-4026-8c6b-f2b6c756dd17") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 13:00:09.923959 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:09.923918    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 13:00:09.924164 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:09.924063    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 13:00:09.924164 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:09.924122    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret podName:26774edd-7404-4111-a22e-da6b38958c85 nodeName:}" failed. No retries permitted until 2026-07-30 13:00:41.92410787 +0000 UTC m=+65.196572245 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret") pod "global-pull-secret-syncer-qbp52" (UID: "26774edd-7404-4111-a22e-da6b38958c85") : object "kube-system"/"original-pull-secret" not registered
Jul 30 13:00:10.177614 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:10.177568    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 13:00:10.177805 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:10.177583    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 13:00:10.177805 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:10.177700    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-769q2" podUID="5f461854-f3bc-4026-8c6b-f2b6c756dd17"
Jul 30 13:00:10.177805 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:10.177729    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 13:00:10.177969 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:10.177853    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbp52" podUID="26774edd-7404-4111-a22e-da6b38958c85"
Jul 30 13:00:10.177969 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:10.177935    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-74gmj" podUID="c083abff-9168-456b-ba24-dd906dc138b2"
Jul 30 13:00:11.544403 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.544375    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-33.ec2.internal" event="NodeReady"
Jul 30 13:00:11.544830 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.544521    2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 13:00:11.589934 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.589855    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7tl52"]
Jul 30 13:00:11.592486 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.592462    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:11.595326 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.595271    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dmnvl"]
Jul 30 13:00:11.596029 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.595961    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 13:00:11.596029 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.595990    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 13:00:11.596029 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.596015    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lv78x\""
Jul 30 13:00:11.597234 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.597215    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dmnvl"
Jul 30 13:00:11.600171 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.600150    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 13:00:11.600277 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.600193    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-r52zx\""
Jul 30 13:00:11.600436 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.600420    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 13:00:11.600617 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.600585    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 13:00:11.604923 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.604837    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7tl52"]
Jul 30 13:00:11.607973 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.607949    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dmnvl"]
Jul 30 13:00:11.738375 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.738333    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/00eda289-36d7-4d10-ab5e-849c4f33b5d7-config-volume\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:11.738552 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.738382    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5p9c\" (UniqueName: \"kubernetes.io/projected/080d61dc-1429-482d-bd36-a448dcd0ee30-kube-api-access-h5p9c\") pod \"ingress-canary-dmnvl\" (UID: \"080d61dc-1429-482d-bd36-a448dcd0ee30\") " pod="openshift-ingress-canary/ingress-canary-dmnvl"
Jul 30 13:00:11.738552 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.738422    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqh7z\" (UniqueName: \"kubernetes.io/projected/00eda289-36d7-4d10-ab5e-849c4f33b5d7-kube-api-access-tqh7z\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:11.738552 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.738468    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00eda289-36d7-4d10-ab5e-849c4f33b5d7-tmp-dir\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:11.738552 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.738508    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/080d61dc-1429-482d-bd36-a448dcd0ee30-cert\") pod \"ingress-canary-dmnvl\" (UID: \"080d61dc-1429-482d-bd36-a448dcd0ee30\") " pod="openshift-ingress-canary/ingress-canary-dmnvl"
Jul 30 13:00:11.738759 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.738565    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:11.839996 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.839892    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/00eda289-36d7-4d10-ab5e-849c4f33b5d7-config-volume\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:11.839996 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.839939    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5p9c\" (UniqueName: \"kubernetes.io/projected/080d61dc-1429-482d-bd36-a448dcd0ee30-kube-api-access-h5p9c\") pod \"ingress-canary-dmnvl\" (UID: \"080d61dc-1429-482d-bd36-a448dcd0ee30\") " pod="openshift-ingress-canary/ingress-canary-dmnvl"
Jul 30 13:00:11.839996 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.839976    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqh7z\" (UniqueName: \"kubernetes.io/projected/00eda289-36d7-4d10-ab5e-849c4f33b5d7-kube-api-access-tqh7z\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:11.840279 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.840004    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00eda289-36d7-4d10-ab5e-849c4f33b5d7-tmp-dir\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:11.840279 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.840026    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/080d61dc-1429-482d-bd36-a448dcd0ee30-cert\") pod \"ingress-canary-dmnvl\" (UID: \"080d61dc-1429-482d-bd36-a448dcd0ee30\") " pod="openshift-ingress-canary/ingress-canary-dmnvl"
Jul 30 13:00:11.840279 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.840061    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:11.840279 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:11.840144    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 13:00:11.840279 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:11.840205    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls podName:00eda289-36d7-4d10-ab5e-849c4f33b5d7 nodeName:}" failed. No retries permitted until 2026-07-30 13:00:12.340187496 +0000 UTC m=+35.612651884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls") pod "dns-default-7tl52" (UID: "00eda289-36d7-4d10-ab5e-849c4f33b5d7") : secret "dns-default-metrics-tls" not found
Jul 30 13:00:11.840524 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.840401    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00eda289-36d7-4d10-ab5e-849c4f33b5d7-tmp-dir\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:11.840684 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.840660    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/00eda289-36d7-4d10-ab5e-849c4f33b5d7-config-volume\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:11.846028 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.846000    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/080d61dc-1429-482d-bd36-a448dcd0ee30-cert\") pod \"ingress-canary-dmnvl\" (UID: \"080d61dc-1429-482d-bd36-a448dcd0ee30\") " pod="openshift-ingress-canary/ingress-canary-dmnvl"
Jul 30 13:00:11.849720 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.849691    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqh7z\" (UniqueName: \"kubernetes.io/projected/00eda289-36d7-4d10-ab5e-849c4f33b5d7-kube-api-access-tqh7z\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:11.849904 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.849876    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5p9c\" (UniqueName: \"kubernetes.io/projected/080d61dc-1429-482d-bd36-a448dcd0ee30-kube-api-access-h5p9c\") pod \"ingress-canary-dmnvl\" (UID: \"080d61dc-1429-482d-bd36-a448dcd0ee30\") " pod="openshift-ingress-canary/ingress-canary-dmnvl"
Jul 30 13:00:11.912893 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:11.912856    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dmnvl"
Jul 30 13:00:12.177733 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:12.177516    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 13:00:12.177972 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:12.177517    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 13:00:12.177972 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:12.177566    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 13:00:12.183918 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:12.183708    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cq62z\""
Jul 30 13:00:12.183918 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:12.183727    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 13:00:12.183918 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:12.183751    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 13:00:12.183918 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:12.183708    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 13:00:12.183918 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:12.183843    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 13:00:12.184280 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:12.184097    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v4mfl\""
Jul 30 13:00:12.343896 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:12.343858    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:12.344076 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:12.344033    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 13:00:12.344128 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:12.344102    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls podName:00eda289-36d7-4d10-ab5e-849c4f33b5d7 nodeName:}" failed. No retries permitted until 2026-07-30 13:00:13.344081756 +0000 UTC m=+36.616546140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls") pod "dns-default-7tl52" (UID: "00eda289-36d7-4d10-ab5e-849c4f33b5d7") : secret "dns-default-metrics-tls" not found
Jul 30 13:00:12.810405 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:12.810377    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dmnvl"]
Jul 30 13:00:12.813432 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:00:12.813405    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod080d61dc_1429_482d_bd36_a448dcd0ee30.slice/crio-1eb18378041250fadf862901cb0e49be9637dd7f7d1a2d8702c8d71eec220537 WatchSource:0}: Error finding container 1eb18378041250fadf862901cb0e49be9637dd7f7d1a2d8702c8d71eec220537: Status 404 returned error can't find the container with id 1eb18378041250fadf862901cb0e49be9637dd7f7d1a2d8702c8d71eec220537
Jul 30 13:00:13.350661 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:13.350621    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:13.350841 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:13.350746    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 13:00:13.350841 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:13.350805    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls podName:00eda289-36d7-4d10-ab5e-849c4f33b5d7 nodeName:}" failed. No retries permitted until 2026-07-30 13:00:15.350788233 +0000 UTC m=+38.623252610 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls") pod "dns-default-7tl52" (UID: "00eda289-36d7-4d10-ab5e-849c4f33b5d7") : secret "dns-default-metrics-tls" not found
Jul 30 13:00:13.412446 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:13.412405    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dmnvl" event={"ID":"080d61dc-1429-482d-bd36-a448dcd0ee30","Type":"ContainerStarted","Data":"1eb18378041250fadf862901cb0e49be9637dd7f7d1a2d8702c8d71eec220537"}
Jul 30 13:00:13.418023 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:13.417985    2557 generic.go:356] "Generic (PLEG): container finished" podID="5972f608-8acf-4650-8b1e-5ad1d2d49bed" containerID="6e8dca6d87e672a49e7756daf72a39cf6b8674da8b3da4d5bf69daaafda9cbe4" exitCode=0
Jul 30 13:00:13.418160 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:13.418031    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7mbgz" event={"ID":"5972f608-8acf-4650-8b1e-5ad1d2d49bed","Type":"ContainerDied","Data":"6e8dca6d87e672a49e7756daf72a39cf6b8674da8b3da4d5bf69daaafda9cbe4"}
Jul 30 13:00:14.423988 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:14.423948    2557 generic.go:356] "Generic (PLEG): container finished" podID="5972f608-8acf-4650-8b1e-5ad1d2d49bed" containerID="9af94d62510955eefca90051b9f6214ef261793e4ad767bd9554be7188a7667c" exitCode=0
Jul 30 13:00:14.424494 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:14.424027    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7mbgz" event={"ID":"5972f608-8acf-4650-8b1e-5ad1d2d49bed","Type":"ContainerDied","Data":"9af94d62510955eefca90051b9f6214ef261793e4ad767bd9554be7188a7667c"}
Jul 30 13:00:15.366074 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:15.366037    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:15.366244 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:15.366219    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 13:00:15.366314 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:15.366304    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls podName:00eda289-36d7-4d10-ab5e-849c4f33b5d7 nodeName:}" failed. No retries permitted until 2026-07-30 13:00:19.366283327 +0000 UTC m=+42.638747708 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls") pod "dns-default-7tl52" (UID: "00eda289-36d7-4d10-ab5e-849c4f33b5d7") : secret "dns-default-metrics-tls" not found
Jul 30 13:00:15.431290 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:15.431249    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7mbgz" event={"ID":"5972f608-8acf-4650-8b1e-5ad1d2d49bed","Type":"ContainerStarted","Data":"151a79b2b3b14164163104b3c14b704f72a5847a43284057d2885190d22fdead"}
Jul 30 13:00:15.455360 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:15.455296    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7mbgz" podStartSLOduration=5.73311445 podStartE2EDuration="37.455272965s" totalImagesPullingTime="31.722158515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 12:59:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:00:15.453771881 +0000 UTC m=+38.726236305" watchObservedRunningTime="2026-07-30 13:00:15.455272965 +0000 UTC m=+38.727737362"
Jul 30 13:00:16.435397 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:16.435360    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dmnvl" event={"ID":"080d61dc-1429-482d-bd36-a448dcd0ee30","Type":"ContainerStarted","Data":"d271e7d5c00daa562ec8de9ee652006f253b2b31c1bbe5172062ac461ec5b432"}
Jul 30 13:00:16.451834 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:16.451778    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dmnvl" podStartSLOduration=2.644738743 podStartE2EDuration="5.451746273s" totalImagesPullingTime="2.80700753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:00:16.451556903 +0000 UTC m=+39.724021300" watchObservedRunningTime="2026-07-30 13:00:16.451746273 +0000 UTC m=+39.724210671"
Jul 30 13:00:19.396548 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:19.396499    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:19.397057 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:19.396702    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 13:00:19.397057 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:19.396785    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls podName:00eda289-36d7-4d10-ab5e-849c4f33b5d7 nodeName:}" failed. No retries permitted until 2026-07-30 13:00:27.396764511 +0000 UTC m=+50.669228899 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls") pod "dns-default-7tl52" (UID: "00eda289-36d7-4d10-ab5e-849c4f33b5d7") : secret "dns-default-metrics-tls" not found
Jul 30 13:00:27.451242 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:27.451208    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:27.451722 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:27.451326    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 13:00:27.451722 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:27.451378    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls podName:00eda289-36d7-4d10-ab5e-849c4f33b5d7 nodeName:}" failed. No retries permitted until 2026-07-30 13:00:43.451364178 +0000 UTC m=+66.723828553 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls") pod "dns-default-7tl52" (UID: "00eda289-36d7-4d10-ab5e-849c4f33b5d7") : secret "dns-default-metrics-tls" not found
Jul 30 13:00:37.410559 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:37.410527    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4q7bg"
Jul 30 13:00:40.966789 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:40.966752    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sbk52"]
Jul 30 13:00:40.969067 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:40.969051    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sbk52"
Jul 30 13:00:40.973586 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:40.973564    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 30 13:00:40.973681 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:40.973613    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2bsjr\""
Jul 30 13:00:40.973681 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:40.973615    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 30 13:00:40.978096 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:40.978078    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sbk52"]
Jul 30 13:00:40.989331 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:40.989309    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-pr87k"]
Jul 30 13:00:40.990955 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:40.990941    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:40.994102 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:40.994080    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 30 13:00:40.994213 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:40.994118    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 13:00:40.994213 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:40.994118    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 13:00:40.994213 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:40.994205    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-hlsmz\""
Jul 30 13:00:40.994440 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:40.994422    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 30 13:00:40.999446 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:40.999426    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 30 13:00:41.002443 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.002424    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-pr87k"]
Jul 30 13:00:41.045462 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.045426    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-tmp\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.045644 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.045502    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-snapshots\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.045644 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.045532    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.045644 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.045549    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-service-ca-bundle\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.045644 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.045572    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-serving-cert\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.045644 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.045627    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn4dq\" (UniqueName: \"kubernetes.io/projected/f9840c97-feb0-49d7-85b0-62a6de93d0b3-kube-api-access-dn4dq\") pod \"volume-data-source-validator-549c6d54cf-sbk52\" (UID: \"f9840c97-feb0-49d7-85b0-62a6de93d0b3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sbk52"
Jul 30 13:00:41.045820 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.045655    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr89c\" (UniqueName: \"kubernetes.io/projected/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-kube-api-access-pr89c\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.076111 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.076080    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"]
Jul 30 13:00:41.077914 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.077895    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.082181 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.082162    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 30 13:00:41.082287 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.082230    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 30 13:00:41.083559 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.083542    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c6xmw\""
Jul 30 13:00:41.083801 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.083786    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 30 13:00:41.090409 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.090387    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 30 13:00:41.097274 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.097240    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"]
Jul 30 13:00:41.146483 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.146446    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-tmp\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.146687 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.146521    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-snapshots\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.146687 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.146551    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.146687 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.146584    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.146687 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.146678    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-service-ca-bundle\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.146908 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.146718    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82174646-9d73-4b26-a17a-6906c0da18de-image-registry-private-configuration\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.146908 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.146759    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-serving-cert\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.146908 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.146865    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82174646-9d73-4b26-a17a-6906c0da18de-registry-certificates\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.146908 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.146896    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82174646-9d73-4b26-a17a-6906c0da18de-installation-pull-secrets\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.147144 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.146931    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn4dq\" (UniqueName: \"kubernetes.io/projected/f9840c97-feb0-49d7-85b0-62a6de93d0b3-kube-api-access-dn4dq\") pod \"volume-data-source-validator-549c6d54cf-sbk52\" (UID: \"f9840c97-feb0-49d7-85b0-62a6de93d0b3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sbk52"
Jul 30 13:00:41.147144 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.146961    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr89c\" (UniqueName: \"kubernetes.io/projected/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-kube-api-access-pr89c\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.147144 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.146987    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82174646-9d73-4b26-a17a-6906c0da18de-trusted-ca\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.147144 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.147011    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mjqs\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-kube-api-access-8mjqs\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.147144 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.147041    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82174646-9d73-4b26-a17a-6906c0da18de-ca-trust-extracted\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.147144 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.147086    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-bound-sa-token\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.147144 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.147123    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-tmp\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.147429 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.147206    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-snapshots\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.147429 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.147254    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-service-ca-bundle\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.147516 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.147484    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.150042 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.150022    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-serving-cert\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.157773 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.157746    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn4dq\" (UniqueName: \"kubernetes.io/projected/f9840c97-feb0-49d7-85b0-62a6de93d0b3-kube-api-access-dn4dq\") pod \"volume-data-source-validator-549c6d54cf-sbk52\" (UID: \"f9840c97-feb0-49d7-85b0-62a6de93d0b3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sbk52"
Jul 30 13:00:41.158083 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.158062    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr89c\" (UniqueName: \"kubernetes.io/projected/5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0-kube-api-access-pr89c\") pod \"insights-operator-5885fdb759-pr87k\" (UID: \"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0\") " pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.248464 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.248373    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.248464 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.248422    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82174646-9d73-4b26-a17a-6906c0da18de-image-registry-private-configuration\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.248464 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.248443    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82174646-9d73-4b26-a17a-6906c0da18de-registry-certificates\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.248750 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:41.248518    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 13:00:41.248750 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:41.248537    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dd57f4fd9-zgq4v: secret "image-registry-tls" not found
Jul 30 13:00:41.248750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.248564    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82174646-9d73-4b26-a17a-6906c0da18de-installation-pull-secrets\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.248750 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:41.248616    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls podName:82174646-9d73-4b26-a17a-6906c0da18de nodeName:}" failed. No retries permitted until 2026-07-30 13:00:41.748585505 +0000 UTC m=+65.021049884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls") pod "image-registry-6dd57f4fd9-zgq4v" (UID: "82174646-9d73-4b26-a17a-6906c0da18de") : secret "image-registry-tls" not found
Jul 30 13:00:41.248750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.248650    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82174646-9d73-4b26-a17a-6906c0da18de-trusted-ca\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.248750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.248670    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mjqs\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-kube-api-access-8mjqs\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.248750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.248691    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82174646-9d73-4b26-a17a-6906c0da18de-ca-trust-extracted\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.249007 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.248803    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-bound-sa-token\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.249411 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.249385    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82174646-9d73-4b26-a17a-6906c0da18de-ca-trust-extracted\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.249509 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.249439    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82174646-9d73-4b26-a17a-6906c0da18de-registry-certificates\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.249775 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.249754    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82174646-9d73-4b26-a17a-6906c0da18de-trusted-ca\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.251390 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.251356    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82174646-9d73-4b26-a17a-6906c0da18de-image-registry-private-configuration\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.251500 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.251419    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82174646-9d73-4b26-a17a-6906c0da18de-installation-pull-secrets\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.260439 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.260415    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-bound-sa-token\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.260636 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.260619    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mjqs\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-kube-api-access-8mjqs\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.277859 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.277831    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sbk52"
Jul 30 13:00:41.299529 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.299505    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-pr87k"
Jul 30 13:00:41.408325 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.408209    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sbk52"]
Jul 30 13:00:41.410738 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:00:41.410712    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9840c97_feb0_49d7_85b0_62a6de93d0b3.slice/crio-fdc33fa25311a3fb999c8f2746e85cfaf44890c7fe1d706cf3de316ae0729693 WatchSource:0}: Error finding container fdc33fa25311a3fb999c8f2746e85cfaf44890c7fe1d706cf3de316ae0729693: Status 404 returned error can't find the container with id fdc33fa25311a3fb999c8f2746e85cfaf44890c7fe1d706cf3de316ae0729693
Jul 30 13:00:41.438179 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.438153    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-pr87k"]
Jul 30 13:00:41.440539 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:00:41.440517    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc557b6_e9d4_4570_aec9_7e2e5f8eebe0.slice/crio-a898954d15eefaa0e1e6a2684540ee7c33972e2ac1e79f5e1bb034e4b86fc0c4 WatchSource:0}: Error finding container a898954d15eefaa0e1e6a2684540ee7c33972e2ac1e79f5e1bb034e4b86fc0c4: Status 404 returned error can't find the container with id a898954d15eefaa0e1e6a2684540ee7c33972e2ac1e79f5e1bb034e4b86fc0c4
Jul 30 13:00:41.490727 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.490690    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-pr87k" event={"ID":"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0","Type":"ContainerStarted","Data":"a898954d15eefaa0e1e6a2684540ee7c33972e2ac1e79f5e1bb034e4b86fc0c4"}
Jul 30 13:00:41.491697 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.491676    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sbk52" event={"ID":"f9840c97-feb0-49d7-85b0-62a6de93d0b3","Type":"ContainerStarted","Data":"fdc33fa25311a3fb999c8f2746e85cfaf44890c7fe1d706cf3de316ae0729693"}
Jul 30 13:00:41.752690 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.752657    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:41.752847 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:41.752778    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 13:00:41.752847 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:41.752790    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dd57f4fd9-zgq4v: secret "image-registry-tls" not found
Jul 30 13:00:41.752847 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:41.752839    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls podName:82174646-9d73-4b26-a17a-6906c0da18de nodeName:}" failed. No retries permitted until 2026-07-30 13:00:42.752826179 +0000 UTC m=+66.025290556 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls") pod "image-registry-6dd57f4fd9-zgq4v" (UID: "82174646-9d73-4b26-a17a-6906c0da18de") : secret "image-registry-tls" not found
Jul 30 13:00:41.853459 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.853425    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs\") pod \"network-metrics-daemon-74gmj\" (UID: \"c083abff-9168-456b-ba24-dd906dc138b2\") " pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 13:00:41.853628 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.853470    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9rzq\" (UniqueName: \"kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq\") pod \"network-check-target-769q2\" (UID: \"5f461854-f3bc-4026-8c6b-f2b6c756dd17\") " pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 13:00:41.856670 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.856653    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 13:00:41.856814 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.856797    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 13:00:41.864032 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:41.864012    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 30 13:00:41.864078 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:41.864067    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs podName:c083abff-9168-456b-ba24-dd906dc138b2 nodeName:}" failed. No retries permitted until 2026-07-30 13:01:45.864053421 +0000 UTC m=+129.136517799 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs") pod "network-metrics-daemon-74gmj" (UID: "c083abff-9168-456b-ba24-dd906dc138b2") : secret "metrics-daemon-secret" not found
Jul 30 13:00:41.866615 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.866584    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 13:00:41.877429 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.877408    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9rzq\" (UniqueName: \"kubernetes.io/projected/5f461854-f3bc-4026-8c6b-f2b6c756dd17-kube-api-access-r9rzq\") pod \"network-check-target-769q2\" (UID: \"5f461854-f3bc-4026-8c6b-f2b6c756dd17\") " pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 13:00:41.892165 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.892147    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cq62z\""
Jul 30 13:00:41.899480 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.899463    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 13:00:41.954243 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.954212    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 13:00:41.957476 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.957456    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 13:00:41.967145 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:41.967126    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26774edd-7404-4111-a22e-da6b38958c85-original-pull-secret\") pod \"global-pull-secret-syncer-qbp52\" (UID: \"26774edd-7404-4111-a22e-da6b38958c85\") " pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 13:00:42.016821 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:42.016618    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-769q2"]
Jul 30 13:00:42.019831 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:00:42.019803    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f461854_f3bc_4026_8c6b_f2b6c756dd17.slice/crio-ba07c4fe8481fd2f09cfbd26deeeafc9783b11b1212da7922412fb8d038ed426 WatchSource:0}: Error finding container ba07c4fe8481fd2f09cfbd26deeeafc9783b11b1212da7922412fb8d038ed426: Status 404 returned error can't find the container with id ba07c4fe8481fd2f09cfbd26deeeafc9783b11b1212da7922412fb8d038ed426
Jul 30 13:00:42.202501 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:42.202461    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbp52"
Jul 30 13:00:42.322353 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:42.322273    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qbp52"]
Jul 30 13:00:42.326811 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:00:42.326781    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26774edd_7404_4111_a22e_da6b38958c85.slice/crio-95a7f489d37c9e6a2a9ba604a5a597311642011cea3dcbe97f557161f28ff755 WatchSource:0}: Error finding container 95a7f489d37c9e6a2a9ba604a5a597311642011cea3dcbe97f557161f28ff755: Status 404 returned error can't find the container with id 95a7f489d37c9e6a2a9ba604a5a597311642011cea3dcbe97f557161f28ff755
Jul 30 13:00:42.496289 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:42.496241    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qbp52" event={"ID":"26774edd-7404-4111-a22e-da6b38958c85","Type":"ContainerStarted","Data":"95a7f489d37c9e6a2a9ba604a5a597311642011cea3dcbe97f557161f28ff755"}
Jul 30 13:00:42.497706 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:42.497663    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-769q2" event={"ID":"5f461854-f3bc-4026-8c6b-f2b6c756dd17","Type":"ContainerStarted","Data":"ba07c4fe8481fd2f09cfbd26deeeafc9783b11b1212da7922412fb8d038ed426"}
Jul 30 13:00:42.761398 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:42.761362    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:42.761586 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:42.761512    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 13:00:42.761586 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:42.761533    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dd57f4fd9-zgq4v: secret "image-registry-tls" not found
Jul 30 13:00:42.761717 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:42.761618    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls podName:82174646-9d73-4b26-a17a-6906c0da18de nodeName:}" failed. No retries permitted until 2026-07-30 13:00:44.761583201 +0000 UTC m=+68.034047591 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls") pod "image-registry-6dd57f4fd9-zgq4v" (UID: "82174646-9d73-4b26-a17a-6906c0da18de") : secret "image-registry-tls" not found
Jul 30 13:00:43.469049 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:43.468427    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:00:43.469049 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:43.468611    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 13:00:43.469049 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:43.468674    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls podName:00eda289-36d7-4d10-ab5e-849c4f33b5d7 nodeName:}" failed. No retries permitted until 2026-07-30 13:01:15.468654804 +0000 UTC m=+98.741119185 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls") pod "dns-default-7tl52" (UID: "00eda289-36d7-4d10-ab5e-849c4f33b5d7") : secret "dns-default-metrics-tls" not found
Jul 30 13:00:43.503833 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:43.503794    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sbk52" event={"ID":"f9840c97-feb0-49d7-85b0-62a6de93d0b3","Type":"ContainerStarted","Data":"b9fe368d32c743b1be347a46eab884136f65750616dd2b607495af81d59ad574"}
Jul 30 13:00:44.508566 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:44.508529    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-pr87k" event={"ID":"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0","Type":"ContainerStarted","Data":"b446dd8e4046e51dac9ed87d22e736cc63d30cff0185d9a9d0675de6efdb8ffb"}
Jul 30 13:00:44.526258 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:44.526203    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sbk52" podStartSLOduration=3.323778843 podStartE2EDuration="4.526182402s" totalImagesPullingTime="1.202403559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:00:43.52323602 +0000 UTC m=+66.795700419" watchObservedRunningTime="2026-07-30 13:00:44.526182402 +0000 UTC m=+67.798646800"
Jul 30 13:00:44.527332 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:44.527289    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-pr87k" podStartSLOduration=2.092589923 podStartE2EDuration="4.527274134s" totalImagesPullingTime="2.434684211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:00:44.525803727 +0000 UTC m=+67.798268160" watchObservedRunningTime="2026-07-30 13:00:44.527274134 +0000 UTC m=+67.799738532"
Jul 30 13:00:44.781010 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:44.780917    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:44.781173 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:44.781097    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 13:00:44.781173 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:44.781121    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dd57f4fd9-zgq4v: secret "image-registry-tls" not found
Jul 30 13:00:44.781291 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:44.781194    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls podName:82174646-9d73-4b26-a17a-6906c0da18de nodeName:}" failed. No retries permitted until 2026-07-30 13:00:48.781172982 +0000 UTC m=+72.053637370 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls") pod "image-registry-6dd57f4fd9-zgq4v" (UID: "82174646-9d73-4b26-a17a-6906c0da18de") : secret "image-registry-tls" not found
Jul 30 13:00:45.414243 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.413058    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t47c8"]
Jul 30 13:00:45.433374 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.433310    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t47c8"]
Jul 30 13:00:45.433615 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.433454    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:45.436321 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.436262    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 13:00:45.436321 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.436305    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 13:00:45.436502 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.436313    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nzhhh\""
Jul 30 13:00:45.488400 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.488366    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:45.488400 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.488407    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsszk\" (UniqueName: \"kubernetes.io/projected/47958414-0ee8-4833-9018-e8785b5ae94f-kube-api-access-bsszk\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:45.488684 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.488438    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:45.488684 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.488464    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/47958414-0ee8-4833-9018-e8785b5ae94f-crio-socket\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:45.488684 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.488611    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/47958414-0ee8-4833-9018-e8785b5ae94f-data-volume\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:45.589509 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.589474    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:45.589977 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.589519    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsszk\" (UniqueName: \"kubernetes.io/projected/47958414-0ee8-4833-9018-e8785b5ae94f-kube-api-access-bsszk\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:45.589977 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:45.589630    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 13:00:45.589977 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.589701    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:45.589977 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:45.589727    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm podName:47958414-0ee8-4833-9018-e8785b5ae94f nodeName:}" failed. No retries permitted until 2026-07-30 13:00:46.089705074 +0000 UTC m=+69.362169448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t47c8" (UID: "47958414-0ee8-4833-9018-e8785b5ae94f") : configmap "kube-rbac-proxy" not found
Jul 30 13:00:45.589977 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.589768    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/47958414-0ee8-4833-9018-e8785b5ae94f-crio-socket\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:45.589977 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:45.589804    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 13:00:45.589977 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.589848    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/47958414-0ee8-4833-9018-e8785b5ae94f-data-volume\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:45.589977 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:45.589869    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls podName:47958414-0ee8-4833-9018-e8785b5ae94f nodeName:}" failed. No retries permitted until 2026-07-30 13:00:46.089851501 +0000 UTC m=+69.362315893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t47c8" (UID: "47958414-0ee8-4833-9018-e8785b5ae94f") : secret "insights-runtime-extractor-tls" not found
Jul 30 13:00:45.589977 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.589903    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/47958414-0ee8-4833-9018-e8785b5ae94f-crio-socket\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:45.590270 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.590076    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/47958414-0ee8-4833-9018-e8785b5ae94f-data-volume\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:45.598584 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:45.598565    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsszk\" (UniqueName: \"kubernetes.io/projected/47958414-0ee8-4833-9018-e8785b5ae94f-kube-api-access-bsszk\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:46.093857 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:46.093818    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:46.093857 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:46.093862    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:46.094113 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:46.093955    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 13:00:46.094113 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:46.093989    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 13:00:46.094113 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:46.094035    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm podName:47958414-0ee8-4833-9018-e8785b5ae94f nodeName:}" failed. No retries permitted until 2026-07-30 13:00:47.094015589 +0000 UTC m=+70.366479984 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t47c8" (UID: "47958414-0ee8-4833-9018-e8785b5ae94f") : configmap "kube-rbac-proxy" not found
Jul 30 13:00:46.094113 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:46.094053    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls podName:47958414-0ee8-4833-9018-e8785b5ae94f nodeName:}" failed. No retries permitted until 2026-07-30 13:00:47.094045059 +0000 UTC m=+70.366509434 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t47c8" (UID: "47958414-0ee8-4833-9018-e8785b5ae94f") : secret "insights-runtime-extractor-tls" not found
Jul 30 13:00:46.146560 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:46.146525    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-zx79q"]
Jul 30 13:00:46.165322 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:46.165289    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-zx79q"]
Jul 30 13:00:46.165499 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:46.165426    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-zx79q"
Jul 30 13:00:46.169307 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:46.169284    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 30 13:00:46.170520 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:46.170501    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 30 13:00:46.170688 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:46.170509    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-4zx7k\""
Jul 30 13:00:46.295980 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:46.295949    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b28jx\" (UniqueName: \"kubernetes.io/projected/226736e9-00a0-4887-b101-fa79c578eee9-kube-api-access-b28jx\") pod \"migrator-6865c9967-zx79q\" (UID: \"226736e9-00a0-4887-b101-fa79c578eee9\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-zx79q"
Jul 30 13:00:46.396976 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:46.396883    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b28jx\" (UniqueName: \"kubernetes.io/projected/226736e9-00a0-4887-b101-fa79c578eee9-kube-api-access-b28jx\") pod \"migrator-6865c9967-zx79q\" (UID: \"226736e9-00a0-4887-b101-fa79c578eee9\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-zx79q"
Jul 30 13:00:46.405337 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:46.405312    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b28jx\" (UniqueName: \"kubernetes.io/projected/226736e9-00a0-4887-b101-fa79c578eee9-kube-api-access-b28jx\") pod \"migrator-6865c9967-zx79q\" (UID: \"226736e9-00a0-4887-b101-fa79c578eee9\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-zx79q"
Jul 30 13:00:46.477318 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:46.477264    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-zx79q"
Jul 30 13:00:47.103560 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:47.103522    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:47.103939 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:47.103586    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:47.103939 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:47.103678    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 13:00:47.103939 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:47.103687    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 13:00:47.103939 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:47.103749    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls podName:47958414-0ee8-4833-9018-e8785b5ae94f nodeName:}" failed. No retries permitted until 2026-07-30 13:00:49.103735808 +0000 UTC m=+72.376200186 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t47c8" (UID: "47958414-0ee8-4833-9018-e8785b5ae94f") : secret "insights-runtime-extractor-tls" not found
Jul 30 13:00:47.103939 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:47.103767    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm podName:47958414-0ee8-4833-9018-e8785b5ae94f nodeName:}" failed. No retries permitted until 2026-07-30 13:00:49.103760293 +0000 UTC m=+72.376224667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t47c8" (UID: "47958414-0ee8-4833-9018-e8785b5ae94f") : configmap "kube-rbac-proxy" not found
Jul 30 13:00:47.379395 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:47.379369    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fng7k_bad62a71-0b34-453b-97c5-bb6ef94cabda/dns-node-resolver/0.log"
Jul 30 13:00:47.396348 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:47.396198    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-zx79q"]
Jul 30 13:00:47.399615 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:00:47.399555    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod226736e9_00a0_4887_b101_fa79c578eee9.slice/crio-b3acdd2e778de7fce23471357ddfcbac8b1c93990dc6637ae9d0f8ea56557741 WatchSource:0}: Error finding container b3acdd2e778de7fce23471357ddfcbac8b1c93990dc6637ae9d0f8ea56557741: Status 404 returned error can't find the container with id b3acdd2e778de7fce23471357ddfcbac8b1c93990dc6637ae9d0f8ea56557741
Jul 30 13:00:47.522428 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:47.522393    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qbp52" event={"ID":"26774edd-7404-4111-a22e-da6b38958c85","Type":"ContainerStarted","Data":"5ea425c8bb3780fbbb5034558e5bbdce090a75c3fccae4a7ab3c3c3056be0f5c"}
Jul 30 13:00:47.523861 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:47.523833    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-zx79q" event={"ID":"226736e9-00a0-4887-b101-fa79c578eee9","Type":"ContainerStarted","Data":"b3acdd2e778de7fce23471357ddfcbac8b1c93990dc6637ae9d0f8ea56557741"}
Jul 30 13:00:47.525301 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:47.525279    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-769q2" event={"ID":"5f461854-f3bc-4026-8c6b-f2b6c756dd17","Type":"ContainerStarted","Data":"04f8fa573c0646a477e009f8de6c9f75f85c67afd69d33ef2b8b89a574dec508"}
Jul 30 13:00:47.525417 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:47.525404    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 13:00:47.543690 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:47.543649    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qbp52" podStartSLOduration=64.592420961 podStartE2EDuration="1m9.543636852s" totalImagesPullingTime="4.951215891s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 12:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:00:47.542264567 +0000 UTC m=+70.814728965" watchObservedRunningTime="2026-07-30 13:00:47.543636852 +0000 UTC m=+70.816101251"
Jul 30 13:00:47.563917 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:47.563872    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-769q2" podStartSLOduration=64.308964519 podStartE2EDuration="1m9.563858027s" totalImagesPullingTime="5.254893508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 12:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:00:47.562750401 +0000 UTC m=+70.835214814" watchObservedRunningTime="2026-07-30 13:00:47.563858027 +0000 UTC m=+70.836322401"
Jul 30 13:00:48.178804 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:48.178773    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5h277_e9b91626-80d1-487b-bdf6-75b892b2dd6b/node-ca/0.log"
Jul 30 13:00:48.817750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:48.817667    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:48.817897 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:48.817823    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 13:00:48.817897 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:48.817842    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dd57f4fd9-zgq4v: secret "image-registry-tls" not found
Jul 30 13:00:48.817897 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:48.817896    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls podName:82174646-9d73-4b26-a17a-6906c0da18de nodeName:}" failed. No retries permitted until 2026-07-30 13:00:56.817882205 +0000 UTC m=+80.090346583 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls") pod "image-registry-6dd57f4fd9-zgq4v" (UID: "82174646-9d73-4b26-a17a-6906c0da18de") : secret "image-registry-tls" not found
Jul 30 13:00:48.978378 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:48.978347    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dmnvl_080d61dc-1429-482d-bd36-a448dcd0ee30/serve-healthcheck-canary/0.log"
Jul 30 13:00:49.119244 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:49.119150    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:49.119244 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:49.119226    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:49.119430 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:49.119321    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 13:00:49.119430 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:49.119353    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 13:00:49.119430 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:49.119379    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls podName:47958414-0ee8-4833-9018-e8785b5ae94f nodeName:}" failed. No retries permitted until 2026-07-30 13:00:53.11936646 +0000 UTC m=+76.391830837 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t47c8" (UID: "47958414-0ee8-4833-9018-e8785b5ae94f") : secret "insights-runtime-extractor-tls" not found
Jul 30 13:00:49.119430 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:49.119402    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm podName:47958414-0ee8-4833-9018-e8785b5ae94f nodeName:}" failed. No retries permitted until 2026-07-30 13:00:53.119389661 +0000 UTC m=+76.391854036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t47c8" (UID: "47958414-0ee8-4833-9018-e8785b5ae94f") : configmap "kube-rbac-proxy" not found
Jul 30 13:00:49.532779 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:49.532744    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-zx79q" event={"ID":"226736e9-00a0-4887-b101-fa79c578eee9","Type":"ContainerStarted","Data":"62efcb3ea27c633b90423222ae42bcf5afcbb0082efffb1cca7ae324ecd864f3"}
Jul 30 13:00:49.533165 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:49.532815    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-zx79q" event={"ID":"226736e9-00a0-4887-b101-fa79c578eee9","Type":"ContainerStarted","Data":"110df4f2f5ed2ca762833e27e227249d3e4299d142f5b6b2258821420c6806ae"}
Jul 30 13:00:49.551068 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:49.551017    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-zx79q" podStartSLOduration=2.467421738 podStartE2EDuration="3.551001272s" totalImagesPullingTime="1.083579534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:00:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:00:49.549616085 +0000 UTC m=+72.822080492" watchObservedRunningTime="2026-07-30 13:00:49.551001272 +0000 UTC m=+72.823465668"
Jul 30 13:00:53.149513 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:53.149456    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:53.149992 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:53.149572    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:00:53.149992 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:53.149575    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 13:00:53.149992 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:53.149664    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm podName:47958414-0ee8-4833-9018-e8785b5ae94f nodeName:}" failed. No retries permitted until 2026-07-30 13:01:01.149648881 +0000 UTC m=+84.422113256 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t47c8" (UID: "47958414-0ee8-4833-9018-e8785b5ae94f") : configmap "kube-rbac-proxy" not found
Jul 30 13:00:53.149992 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:53.149724    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 13:00:53.149992 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:00:53.149783    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls podName:47958414-0ee8-4833-9018-e8785b5ae94f nodeName:}" failed. No retries permitted until 2026-07-30 13:01:01.149767395 +0000 UTC m=+84.422231775 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t47c8" (UID: "47958414-0ee8-4833-9018-e8785b5ae94f") : secret "insights-runtime-extractor-tls" not found
Jul 30 13:00:56.877846 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:56.877798    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:56.880102 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:56.880084    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls\") pod \"image-registry-6dd57f4fd9-zgq4v\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") " pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:56.986941 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:56.986887    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:57.113745 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:57.113716    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"]
Jul 30 13:00:57.116892 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:00:57.116865    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82174646_9d73_4b26_a17a_6906c0da18de.slice/crio-ef239f0c72f471b9729ee1187ffa688db94a3189b4a1938a5481c700a27ef01b WatchSource:0}: Error finding container ef239f0c72f471b9729ee1187ffa688db94a3189b4a1938a5481c700a27ef01b: Status 404 returned error can't find the container with id ef239f0c72f471b9729ee1187ffa688db94a3189b4a1938a5481c700a27ef01b
Jul 30 13:00:57.557162 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:57.557123    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v" event={"ID":"82174646-9d73-4b26-a17a-6906c0da18de","Type":"ContainerStarted","Data":"75215cbff0672550e6f4df532aa890f75495c698d57ad75cecb9eb3c85392a67"}
Jul 30 13:00:57.557367 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:57.557201    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v" event={"ID":"82174646-9d73-4b26-a17a-6906c0da18de","Type":"ContainerStarted","Data":"ef239f0c72f471b9729ee1187ffa688db94a3189b4a1938a5481c700a27ef01b"}
Jul 30 13:00:57.557367 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:57.557237    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:00:57.580092 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:00:57.580044    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v" podStartSLOduration=16.580030185 podStartE2EDuration="16.580030185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:00:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:00:57.577997805 +0000 UTC m=+80.850462202" watchObservedRunningTime="2026-07-30 13:00:57.580030185 +0000 UTC m=+80.852494581"
Jul 30 13:01:01.213948 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:01.213818    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:01:01.213948 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:01.213878    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:01:01.214380 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:01.214363    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47958414-0ee8-4833-9018-e8785b5ae94f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:01:01.216101 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:01.216080    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47958414-0ee8-4833-9018-e8785b5ae94f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t47c8\" (UID: \"47958414-0ee8-4833-9018-e8785b5ae94f\") " pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:01:01.345085 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:01.345051    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t47c8"
Jul 30 13:01:01.473879 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:01.473797    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t47c8"]
Jul 30 13:01:01.477051 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:01:01.477020    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47958414_0ee8_4833_9018_e8785b5ae94f.slice/crio-10ad7c7d4cb49cf7da83ac5cac021fea72ca93f1ea2b71c9951c0a59fce62245 WatchSource:0}: Error finding container 10ad7c7d4cb49cf7da83ac5cac021fea72ca93f1ea2b71c9951c0a59fce62245: Status 404 returned error can't find the container with id 10ad7c7d4cb49cf7da83ac5cac021fea72ca93f1ea2b71c9951c0a59fce62245
Jul 30 13:01:01.570824 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:01.570792    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t47c8" event={"ID":"47958414-0ee8-4833-9018-e8785b5ae94f","Type":"ContainerStarted","Data":"10ad7c7d4cb49cf7da83ac5cac021fea72ca93f1ea2b71c9951c0a59fce62245"}
Jul 30 13:01:07.593755 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:07.593716    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t47c8" event={"ID":"47958414-0ee8-4833-9018-e8785b5ae94f","Type":"ContainerStarted","Data":"bca1e4a779a764503fd7ec12681196e6c6b99226e8cb2a819cd00b19af5a135b"}
Jul 30 13:01:10.605757 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:10.605724    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t47c8" event={"ID":"47958414-0ee8-4833-9018-e8785b5ae94f","Type":"ContainerStarted","Data":"06184a572c57c7a63a7f3e154161830c0ddfd13535904ddb676604a0706d0ba3"}
Jul 30 13:01:10.949320 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:10.949283    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598b796c88-xmhd2"]
Jul 30 13:01:10.952422 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:10.952404    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598b796c88-xmhd2"
Jul 30 13:01:10.955563 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:10.955541    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 30 13:01:10.956823 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:10.956804    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-bz84c\""
Jul 30 13:01:10.956927 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:10.956863    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 30 13:01:10.957011 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:10.956990    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 30 13:01:10.957133 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:10.957118    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 30 13:01:10.969739 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:10.969712    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598b796c88-xmhd2"]
Jul 30 13:01:11.003950 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.003917    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"]
Jul 30 13:01:11.038704 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.038676    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-db6b6c6d9-72cq9"]
Jul 30 13:01:11.041938 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.041914    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.058704 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.058668    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db6b6c6d9-72cq9"]
Jul 30 13:01:11.096345 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.096310    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ed7c293-8209-439f-8498-de46a8d58cf3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-598b796c88-xmhd2\" (UID: \"1ed7c293-8209-439f-8498-de46a8d58cf3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598b796c88-xmhd2"
Jul 30 13:01:11.096650 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.096375    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfgfj\" (UniqueName: \"kubernetes.io/projected/1ed7c293-8209-439f-8498-de46a8d58cf3-kube-api-access-wfgfj\") pod \"managed-serviceaccount-addon-agent-598b796c88-xmhd2\" (UID: \"1ed7c293-8209-439f-8498-de46a8d58cf3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598b796c88-xmhd2"
Jul 30 13:01:11.197325 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.197295    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8c98414-472d-47b8-9b14-275a45e8f39f-trusted-ca\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.197474 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.197331    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8c98414-472d-47b8-9b14-275a45e8f39f-installation-pull-secrets\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.197474 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.197356    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ed7c293-8209-439f-8498-de46a8d58cf3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-598b796c88-xmhd2\" (UID: \"1ed7c293-8209-439f-8498-de46a8d58cf3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598b796c88-xmhd2"
Jul 30 13:01:11.197474 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.197459    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8c98414-472d-47b8-9b14-275a45e8f39f-image-registry-private-configuration\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.197619 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.197491    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfgfj\" (UniqueName: \"kubernetes.io/projected/1ed7c293-8209-439f-8498-de46a8d58cf3-kube-api-access-wfgfj\") pod \"managed-serviceaccount-addon-agent-598b796c88-xmhd2\" (UID: \"1ed7c293-8209-439f-8498-de46a8d58cf3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598b796c88-xmhd2"
Jul 30 13:01:11.197619 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.197517    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8c98414-472d-47b8-9b14-275a45e8f39f-registry-certificates\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.197619 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.197547    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8c98414-472d-47b8-9b14-275a45e8f39f-registry-tls\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.197619 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.197566    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8c98414-472d-47b8-9b14-275a45e8f39f-ca-trust-extracted\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.197619 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.197592    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p59kh\" (UniqueName: \"kubernetes.io/projected/a8c98414-472d-47b8-9b14-275a45e8f39f-kube-api-access-p59kh\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.197791 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.197641    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8c98414-472d-47b8-9b14-275a45e8f39f-bound-sa-token\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.199822 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.199763    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ed7c293-8209-439f-8498-de46a8d58cf3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-598b796c88-xmhd2\" (UID: \"1ed7c293-8209-439f-8498-de46a8d58cf3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598b796c88-xmhd2"
Jul 30 13:01:11.207642 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.207617    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfgfj\" (UniqueName: \"kubernetes.io/projected/1ed7c293-8209-439f-8498-de46a8d58cf3-kube-api-access-wfgfj\") pod \"managed-serviceaccount-addon-agent-598b796c88-xmhd2\" (UID: \"1ed7c293-8209-439f-8498-de46a8d58cf3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598b796c88-xmhd2"
Jul 30 13:01:11.272847 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.272815    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598b796c88-xmhd2"
Jul 30 13:01:11.298068 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.298033    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p59kh\" (UniqueName: \"kubernetes.io/projected/a8c98414-472d-47b8-9b14-275a45e8f39f-kube-api-access-p59kh\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.298289 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.298082    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8c98414-472d-47b8-9b14-275a45e8f39f-bound-sa-token\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.298289 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.298123    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8c98414-472d-47b8-9b14-275a45e8f39f-trusted-ca\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.298289 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.298150    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8c98414-472d-47b8-9b14-275a45e8f39f-installation-pull-secrets\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.298289 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.298188    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8c98414-472d-47b8-9b14-275a45e8f39f-image-registry-private-configuration\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.298289 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.298241    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8c98414-472d-47b8-9b14-275a45e8f39f-registry-certificates\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.298289 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.298280    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8c98414-472d-47b8-9b14-275a45e8f39f-registry-tls\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.298515 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.298310    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8c98414-472d-47b8-9b14-275a45e8f39f-ca-trust-extracted\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.300270 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.298762    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8c98414-472d-47b8-9b14-275a45e8f39f-ca-trust-extracted\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.300270 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.299226    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8c98414-472d-47b8-9b14-275a45e8f39f-registry-certificates\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.300270 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.299687    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8c98414-472d-47b8-9b14-275a45e8f39f-trusted-ca\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.301464 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.301439    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8c98414-472d-47b8-9b14-275a45e8f39f-installation-pull-secrets\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.301560 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.301548    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8c98414-472d-47b8-9b14-275a45e8f39f-image-registry-private-configuration\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.301979 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.301949    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8c98414-472d-47b8-9b14-275a45e8f39f-registry-tls\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.312480 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.312451    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8c98414-472d-47b8-9b14-275a45e8f39f-bound-sa-token\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.312834 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.312806    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p59kh\" (UniqueName: \"kubernetes.io/projected/a8c98414-472d-47b8-9b14-275a45e8f39f-kube-api-access-p59kh\") pod \"image-registry-db6b6c6d9-72cq9\" (UID: \"a8c98414-472d-47b8-9b14-275a45e8f39f\") " pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.352249 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.352216    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.427554 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.427526    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598b796c88-xmhd2"]
Jul 30 13:01:11.430852 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:01:11.430824    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ed7c293_8209_439f_8498_de46a8d58cf3.slice/crio-67350e8014fc7fd3335dedc2f3b0b1fcb6fce9689c4312330155cdf9b027d7da WatchSource:0}: Error finding container 67350e8014fc7fd3335dedc2f3b0b1fcb6fce9689c4312330155cdf9b027d7da: Status 404 returned error can't find the container with id 67350e8014fc7fd3335dedc2f3b0b1fcb6fce9689c4312330155cdf9b027d7da
Jul 30 13:01:11.503578 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.503496    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db6b6c6d9-72cq9"]
Jul 30 13:01:11.506495 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:01:11.506465    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c98414_472d_47b8_9b14_275a45e8f39f.slice/crio-c7874700e23c2054024bd1877e9a76684c6fe4aacfd0d624e8d5bf6a6b124b45 WatchSource:0}: Error finding container c7874700e23c2054024bd1877e9a76684c6fe4aacfd0d624e8d5bf6a6b124b45: Status 404 returned error can't find the container with id c7874700e23c2054024bd1877e9a76684c6fe4aacfd0d624e8d5bf6a6b124b45
Jul 30 13:01:11.611312 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.611276    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598b796c88-xmhd2" event={"ID":"1ed7c293-8209-439f-8498-de46a8d58cf3","Type":"ContainerStarted","Data":"67350e8014fc7fd3335dedc2f3b0b1fcb6fce9689c4312330155cdf9b027d7da"}
Jul 30 13:01:11.612968 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.612943    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9" event={"ID":"a8c98414-472d-47b8-9b14-275a45e8f39f","Type":"ContainerStarted","Data":"3f54723dc8068c401d430cc7b8046104334ef4957b5369cf387e51d6ee81c310"}
Jul 30 13:01:11.613089 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.612999    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9" event={"ID":"a8c98414-472d-47b8-9b14-275a45e8f39f","Type":"ContainerStarted","Data":"c7874700e23c2054024bd1877e9a76684c6fe4aacfd0d624e8d5bf6a6b124b45"}
Jul 30 13:01:11.613143 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.613100    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:11.635877 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:11.635818    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9" podStartSLOduration=0.635794599 podStartE2EDuration="635.794599ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:01:11.634366452 +0000 UTC m=+94.906830851" watchObservedRunningTime="2026-07-30 13:01:11.635794599 +0000 UTC m=+94.908258992"
Jul 30 13:01:15.535608 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:15.535563    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:01:15.538062 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:15.538041    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00eda289-36d7-4d10-ab5e-849c4f33b5d7-metrics-tls\") pod \"dns-default-7tl52\" (UID: \"00eda289-36d7-4d10-ab5e-849c4f33b5d7\") " pod="openshift-dns/dns-default-7tl52"
Jul 30 13:01:15.628781 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:15.628684    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598b796c88-xmhd2" event={"ID":"1ed7c293-8209-439f-8498-de46a8d58cf3","Type":"ContainerStarted","Data":"e8c617f16edc32255e249028c064cc7918bc408fd5836216abe46befa31b29e8"}
Jul 30 13:01:15.630783 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:15.630758    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t47c8" event={"ID":"47958414-0ee8-4833-9018-e8785b5ae94f","Type":"ContainerStarted","Data":"01ee4145d38c496a2d6d26641224f287a6355bc6ddcd90dc288e81c02738260f"}
Jul 30 13:01:15.654207 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:15.654160    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598b796c88-xmhd2" podStartSLOduration=1.748912357 podStartE2EDuration="5.654145501s" totalImagesPullingTime="3.905233144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:01:15.65279331 +0000 UTC m=+98.925257706" watchObservedRunningTime="2026-07-30 13:01:15.654145501 +0000 UTC m=+98.926609889"
Jul 30 13:01:15.671869 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:15.671826    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-t47c8" podStartSLOduration=17.001648093 podStartE2EDuration="30.671811287s" totalImagesPullingTime="13.670163194s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 13:00:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:01:15.671153504 +0000 UTC m=+98.943617904" watchObservedRunningTime="2026-07-30 13:01:15.671811287 +0000 UTC m=+98.944275684"
Jul 30 13:01:15.809083 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:15.809053    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lv78x\""
Jul 30 13:01:15.816778 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:15.816760    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7tl52"
Jul 30 13:01:15.938822 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:15.938790    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7tl52"]
Jul 30 13:01:15.941458 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:01:15.941433    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00eda289_36d7_4d10_ab5e_849c4f33b5d7.slice/crio-16df53d94a5abcd319317c84496e08c78de66b3a8e70d964ac035705061ed914 WatchSource:0}: Error finding container 16df53d94a5abcd319317c84496e08c78de66b3a8e70d964ac035705061ed914: Status 404 returned error can't find the container with id 16df53d94a5abcd319317c84496e08c78de66b3a8e70d964ac035705061ed914
Jul 30 13:01:16.638673 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:16.638633    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7tl52" event={"ID":"00eda289-36d7-4d10-ab5e-849c4f33b5d7","Type":"ContainerStarted","Data":"16df53d94a5abcd319317c84496e08c78de66b3a8e70d964ac035705061ed914"}
Jul 30 13:01:17.643452 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:17.643417    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7tl52" event={"ID":"00eda289-36d7-4d10-ab5e-849c4f33b5d7","Type":"ContainerStarted","Data":"79516108206b9260ffae96cd47fd2d874f611465a5cb05d7aa55c6415801ac62"}
Jul 30 13:01:17.643836 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:17.643469    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7tl52" event={"ID":"00eda289-36d7-4d10-ab5e-849c4f33b5d7","Type":"ContainerStarted","Data":"2ebab2dda1e5145029848e0d4a35e53b90b6a4a4e10f9feddbb266a311345345"}
Jul 30 13:01:17.643836 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:17.643542    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7tl52"
Jul 30 13:01:18.529574 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:18.529541    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-769q2"
Jul 30 13:01:18.546064 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:18.546010    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7tl52" podStartSLOduration=66.058742336 podStartE2EDuration="1m7.545989812s" totalImagesPullingTime="1.487247476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:01:17.673809113 +0000 UTC m=+100.946273522" watchObservedRunningTime="2026-07-30 13:01:18.545989812 +0000 UTC m=+101.818454212"
Jul 30 13:01:21.009154 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:21.009117    2557 patch_prober.go:28] interesting pod/image-registry-6dd57f4fd9-zgq4v container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 30 13:01:21.009527 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:21.009185    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v" podUID="82174646-9d73-4b26-a17a-6906c0da18de" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 13:01:27.651090 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:27.651060    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7tl52"
Jul 30 13:01:31.008353 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:31.008325    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:01:31.358252 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:31.358159    2557 patch_prober.go:28] interesting pod/image-registry-db6b6c6d9-72cq9 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 30 13:01:31.358252 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:31.358214    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9" podUID="a8c98414-472d-47b8-9b14-275a45e8f39f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 13:01:32.622230 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:32.622203    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-db6b6c6d9-72cq9"
Jul 30 13:01:36.026045 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.025981    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v" podUID="82174646-9d73-4b26-a17a-6906c0da18de" containerName="registry" containerID="cri-o://75215cbff0672550e6f4df532aa890f75495c698d57ad75cecb9eb3c85392a67" gracePeriod=30
Jul 30 13:01:36.257235 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.257204    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:01:36.293508 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.293437    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82174646-9d73-4b26-a17a-6906c0da18de-installation-pull-secrets\") pod \"82174646-9d73-4b26-a17a-6906c0da18de\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") "
Jul 30 13:01:36.293508 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.293474    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82174646-9d73-4b26-a17a-6906c0da18de-image-registry-private-configuration\") pod \"82174646-9d73-4b26-a17a-6906c0da18de\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") "
Jul 30 13:01:36.293508 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.293502    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mjqs\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-kube-api-access-8mjqs\") pod \"82174646-9d73-4b26-a17a-6906c0da18de\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") "
Jul 30 13:01:36.293773 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.293526    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82174646-9d73-4b26-a17a-6906c0da18de-registry-certificates\") pod \"82174646-9d73-4b26-a17a-6906c0da18de\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") "
Jul 30 13:01:36.293773 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.293560    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82174646-9d73-4b26-a17a-6906c0da18de-trusted-ca\") pod \"82174646-9d73-4b26-a17a-6906c0da18de\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") "
Jul 30 13:01:36.293773 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.293611    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82174646-9d73-4b26-a17a-6906c0da18de-ca-trust-extracted\") pod \"82174646-9d73-4b26-a17a-6906c0da18de\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") "
Jul 30 13:01:36.293773 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.293640    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-bound-sa-token\") pod \"82174646-9d73-4b26-a17a-6906c0da18de\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") "
Jul 30 13:01:36.293773 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.293665    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls\") pod \"82174646-9d73-4b26-a17a-6906c0da18de\" (UID: \"82174646-9d73-4b26-a17a-6906c0da18de\") "
Jul 30 13:01:36.294134 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.294105    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82174646-9d73-4b26-a17a-6906c0da18de-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "82174646-9d73-4b26-a17a-6906c0da18de" (UID: "82174646-9d73-4b26-a17a-6906c0da18de"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:01:36.294240 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.294159    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82174646-9d73-4b26-a17a-6906c0da18de-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "82174646-9d73-4b26-a17a-6906c0da18de" (UID: "82174646-9d73-4b26-a17a-6906c0da18de"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:01:36.296292 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.296263    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "82174646-9d73-4b26-a17a-6906c0da18de" (UID: "82174646-9d73-4b26-a17a-6906c0da18de"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:01:36.296292 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.296284    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "82174646-9d73-4b26-a17a-6906c0da18de" (UID: "82174646-9d73-4b26-a17a-6906c0da18de"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:01:36.296463 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.296326    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82174646-9d73-4b26-a17a-6906c0da18de-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "82174646-9d73-4b26-a17a-6906c0da18de" (UID: "82174646-9d73-4b26-a17a-6906c0da18de"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 13:01:36.296463 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.296411    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82174646-9d73-4b26-a17a-6906c0da18de-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "82174646-9d73-4b26-a17a-6906c0da18de" (UID: "82174646-9d73-4b26-a17a-6906c0da18de"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 13:01:36.297880 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.297857    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-kube-api-access-8mjqs" (OuterVolumeSpecName: "kube-api-access-8mjqs") pod "82174646-9d73-4b26-a17a-6906c0da18de" (UID: "82174646-9d73-4b26-a17a-6906c0da18de"). InnerVolumeSpecName "kube-api-access-8mjqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:01:36.303242 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.303218    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82174646-9d73-4b26-a17a-6906c0da18de-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "82174646-9d73-4b26-a17a-6906c0da18de" (UID: "82174646-9d73-4b26-a17a-6906c0da18de"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:01:36.394291 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.394259    2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82174646-9d73-4b26-a17a-6906c0da18de-installation-pull-secrets\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:01:36.394291 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.394287    2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82174646-9d73-4b26-a17a-6906c0da18de-image-registry-private-configuration\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:01:36.394291 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.394299    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mjqs\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-kube-api-access-8mjqs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:01:36.394471 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.394307    2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82174646-9d73-4b26-a17a-6906c0da18de-registry-certificates\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:01:36.394471 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.394317    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82174646-9d73-4b26-a17a-6906c0da18de-trusted-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:01:36.394471 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.394325    2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82174646-9d73-4b26-a17a-6906c0da18de-ca-trust-extracted\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:01:36.394471 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.394333    2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-bound-sa-token\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:01:36.394471 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.394341    2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82174646-9d73-4b26-a17a-6906c0da18de-registry-tls\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:01:36.707331 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.707295    2557 generic.go:356] "Generic (PLEG): container finished" podID="82174646-9d73-4b26-a17a-6906c0da18de" containerID="75215cbff0672550e6f4df532aa890f75495c698d57ad75cecb9eb3c85392a67" exitCode=0
Jul 30 13:01:36.707521 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.707341    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"
Jul 30 13:01:36.707521 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.707385    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v" event={"ID":"82174646-9d73-4b26-a17a-6906c0da18de","Type":"ContainerDied","Data":"75215cbff0672550e6f4df532aa890f75495c698d57ad75cecb9eb3c85392a67"}
Jul 30 13:01:36.707521 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.707452    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dd57f4fd9-zgq4v" event={"ID":"82174646-9d73-4b26-a17a-6906c0da18de","Type":"ContainerDied","Data":"ef239f0c72f471b9729ee1187ffa688db94a3189b4a1938a5481c700a27ef01b"}
Jul 30 13:01:36.707521 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.707480    2557 scope.go:117] "RemoveContainer" containerID="75215cbff0672550e6f4df532aa890f75495c698d57ad75cecb9eb3c85392a67"
Jul 30 13:01:36.715653 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.715635    2557 scope.go:117] "RemoveContainer" containerID="75215cbff0672550e6f4df532aa890f75495c698d57ad75cecb9eb3c85392a67"
Jul 30 13:01:36.715901 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:01:36.715881    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75215cbff0672550e6f4df532aa890f75495c698d57ad75cecb9eb3c85392a67\": container with ID starting with 75215cbff0672550e6f4df532aa890f75495c698d57ad75cecb9eb3c85392a67 not found: ID does not exist" containerID="75215cbff0672550e6f4df532aa890f75495c698d57ad75cecb9eb3c85392a67"
Jul 30 13:01:36.715975 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.715914    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75215cbff0672550e6f4df532aa890f75495c698d57ad75cecb9eb3c85392a67"} err="failed to get container status \"75215cbff0672550e6f4df532aa890f75495c698d57ad75cecb9eb3c85392a67\": rpc error: code = NotFound desc = could not find container \"75215cbff0672550e6f4df532aa890f75495c698d57ad75cecb9eb3c85392a67\": container with ID starting with 75215cbff0672550e6f4df532aa890f75495c698d57ad75cecb9eb3c85392a67 not found: ID does not exist"
Jul 30 13:01:36.728008 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.727985    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"]
Jul 30 13:01:36.735760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:36.735736    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6dd57f4fd9-zgq4v"]
Jul 30 13:01:37.181860 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:37.181827    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82174646-9d73-4b26-a17a-6906c0da18de" path="/var/lib/kubelet/pods/82174646-9d73-4b26-a17a-6906c0da18de/volumes"
Jul 30 13:01:40.066319 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.066288    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"]
Jul 30 13:01:40.066844 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.066640    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82174646-9d73-4b26-a17a-6906c0da18de" containerName="registry"
Jul 30 13:01:40.066844 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.066658    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="82174646-9d73-4b26-a17a-6906c0da18de" containerName="registry"
Jul 30 13:01:40.066844 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.066717    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="82174646-9d73-4b26-a17a-6906c0da18de" containerName="registry"
Jul 30 13:01:40.154439 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.154400    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"]
Jul 30 13:01:40.154631 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.154451    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7k62v"]
Jul 30 13:01:40.154631 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.154525    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"
Jul 30 13:01:40.157467 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.157445    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 30 13:01:40.157467 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.157465    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 13:01:40.157684 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.157518    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 13:01:40.157684 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.157553    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 30 13:01:40.157808 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.157795    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-q7m28\""
Jul 30 13:01:40.158855 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.158834    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 13:01:40.180140 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.180120    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.182865 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.182845    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2xgst\""
Jul 30 13:01:40.182954 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.182880    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 13:01:40.183021 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.182962    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 13:01:40.183021 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.182974    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 13:01:40.221092 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.221069    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.221183 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.221099    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c136d17d-f123-49f0-aef4-990cfdea139c-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-gg6kv\" (UID: \"c136d17d-f123-49f0-aef4-990cfdea139c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"
Jul 30 13:01:40.221183 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.221122    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-947l9\" (UniqueName: \"kubernetes.io/projected/c136d17d-f123-49f0-aef4-990cfdea139c-kube-api-access-947l9\") pod \"openshift-state-metrics-b8f49f5c6-gg6kv\" (UID: \"c136d17d-f123-49f0-aef4-990cfdea139c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"
Jul 30 13:01:40.221183 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.221150    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-accelerators-collector-config\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.221183 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.221172    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpjmm\" (UniqueName: \"kubernetes.io/projected/c9c400f7-8b1b-4394-8226-fc7e317de93a-kube-api-access-bpjmm\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.221332 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.221208    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-tls\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.221332 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.221234    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9c400f7-8b1b-4394-8226-fc7e317de93a-sys\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.221332 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.221251    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-wtmp\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.221332 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.221280    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c136d17d-f123-49f0-aef4-990cfdea139c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-gg6kv\" (UID: \"c136d17d-f123-49f0-aef4-990cfdea139c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"
Jul 30 13:01:40.221332 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.221302    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c9c400f7-8b1b-4394-8226-fc7e317de93a-root\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.221494 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.221360    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9c400f7-8b1b-4394-8226-fc7e317de93a-metrics-client-ca\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.221494 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.221398    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-textfile\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.221494 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.221418    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c136d17d-f123-49f0-aef4-990cfdea139c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-gg6kv\" (UID: \"c136d17d-f123-49f0-aef4-990cfdea139c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"
Jul 30 13:01:40.322334 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322276    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9c400f7-8b1b-4394-8226-fc7e317de93a-metrics-client-ca\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.322334 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322311    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-textfile\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.322334 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322332    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c136d17d-f123-49f0-aef4-990cfdea139c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-gg6kv\" (UID: \"c136d17d-f123-49f0-aef4-990cfdea139c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"
Jul 30 13:01:40.322515 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322369    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.322515 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322387    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c136d17d-f123-49f0-aef4-990cfdea139c-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-gg6kv\" (UID: \"c136d17d-f123-49f0-aef4-990cfdea139c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"
Jul 30 13:01:40.322515 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322408    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-947l9\" (UniqueName: \"kubernetes.io/projected/c136d17d-f123-49f0-aef4-990cfdea139c-kube-api-access-947l9\") pod \"openshift-state-metrics-b8f49f5c6-gg6kv\" (UID: \"c136d17d-f123-49f0-aef4-990cfdea139c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"
Jul 30 13:01:40.322515 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322433    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-accelerators-collector-config\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.322515 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322456    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpjmm\" (UniqueName: \"kubernetes.io/projected/c9c400f7-8b1b-4394-8226-fc7e317de93a-kube-api-access-bpjmm\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.322770 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322708    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-textfile\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.322826 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322796    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-tls\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.322877 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322830    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9c400f7-8b1b-4394-8226-fc7e317de93a-sys\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.322877 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322853    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-wtmp\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.322974 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322907    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c136d17d-f123-49f0-aef4-990cfdea139c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-gg6kv\" (UID: \"c136d17d-f123-49f0-aef4-990cfdea139c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"
Jul 30 13:01:40.322974 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:01:40.322927    2557 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 30 13:01:40.323076 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322970    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9c400f7-8b1b-4394-8226-fc7e317de93a-metrics-client-ca\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.323076 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322908    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9c400f7-8b1b-4394-8226-fc7e317de93a-sys\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.323076 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:01:40.322998    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-tls podName:c9c400f7-8b1b-4394-8226-fc7e317de93a nodeName:}" failed. No retries permitted until 2026-07-30 13:01:40.822981155 +0000 UTC m=+124.095445545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-tls") pod "node-exporter-7k62v" (UID: "c9c400f7-8b1b-4394-8226-fc7e317de93a") : secret "node-exporter-tls" not found
Jul 30 13:01:40.323076 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322976    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c9c400f7-8b1b-4394-8226-fc7e317de93a-root\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.323076 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.322935    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c9c400f7-8b1b-4394-8226-fc7e317de93a-root\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.323076 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.323026    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-wtmp\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.323076 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.323031    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-accelerators-collector-config\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.323657 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.323638    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c136d17d-f123-49f0-aef4-990cfdea139c-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-gg6kv\" (UID: \"c136d17d-f123-49f0-aef4-990cfdea139c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"
Jul 30 13:01:40.324826 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.324810    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.324905 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.324858    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c136d17d-f123-49f0-aef4-990cfdea139c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-gg6kv\" (UID: \"c136d17d-f123-49f0-aef4-990cfdea139c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"
Jul 30 13:01:40.324966 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.324946    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c136d17d-f123-49f0-aef4-990cfdea139c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-gg6kv\" (UID: \"c136d17d-f123-49f0-aef4-990cfdea139c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"
Jul 30 13:01:40.333306 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.333277    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpjmm\" (UniqueName: \"kubernetes.io/projected/c9c400f7-8b1b-4394-8226-fc7e317de93a-kube-api-access-bpjmm\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.333849 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.333826    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-947l9\" (UniqueName: \"kubernetes.io/projected/c136d17d-f123-49f0-aef4-990cfdea139c-kube-api-access-947l9\") pod \"openshift-state-metrics-b8f49f5c6-gg6kv\" (UID: \"c136d17d-f123-49f0-aef4-990cfdea139c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"
Jul 30 13:01:40.463100 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.463060    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"
Jul 30 13:01:40.653137 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.653095    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv"]
Jul 30 13:01:40.655701 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:01:40.655674    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc136d17d_f123_49f0_aef4_990cfdea139c.slice/crio-6bd2d847254284cab015f1b820d853ae78513dcb89f6b9f3ab08dadb153c63af WatchSource:0}: Error finding container 6bd2d847254284cab015f1b820d853ae78513dcb89f6b9f3ab08dadb153c63af: Status 404 returned error can't find the container with id 6bd2d847254284cab015f1b820d853ae78513dcb89f6b9f3ab08dadb153c63af
Jul 30 13:01:40.725588 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.725541    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv" event={"ID":"c136d17d-f123-49f0-aef4-990cfdea139c","Type":"ContainerStarted","Data":"6bd2d847254284cab015f1b820d853ae78513dcb89f6b9f3ab08dadb153c63af"}
Jul 30 13:01:40.827750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.827696    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-tls\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:40.829980 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:40.829962    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c9c400f7-8b1b-4394-8226-fc7e317de93a-node-exporter-tls\") pod \"node-exporter-7k62v\" (UID: \"c9c400f7-8b1b-4394-8226-fc7e317de93a\") " pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:41.088074 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:41.088001    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7k62v"
Jul 30 13:01:41.124184 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:01:41.124151    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9c400f7_8b1b_4394_8226_fc7e317de93a.slice/crio-b081f278e559b8cfc8b1887416593d73f0003f38536e90fd285e0c8e415b30ea WatchSource:0}: Error finding container b081f278e559b8cfc8b1887416593d73f0003f38536e90fd285e0c8e415b30ea: Status 404 returned error can't find the container with id b081f278e559b8cfc8b1887416593d73f0003f38536e90fd285e0c8e415b30ea
Jul 30 13:01:41.730155 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:41.730119    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv" event={"ID":"c136d17d-f123-49f0-aef4-990cfdea139c","Type":"ContainerStarted","Data":"6ee7dcec4485d97192a4d696ba9a88b5548fdef822f4218cee42539aa85ee129"}
Jul 30 13:01:41.730428 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:41.730186    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv" event={"ID":"c136d17d-f123-49f0-aef4-990cfdea139c","Type":"ContainerStarted","Data":"96010063030e3cf939c7884c1bc0ebc298bdbc7d510c8d2c75b55f3e4ecc9233"}
Jul 30 13:01:41.731229 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:41.731205    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7k62v" event={"ID":"c9c400f7-8b1b-4394-8226-fc7e317de93a","Type":"ContainerStarted","Data":"b081f278e559b8cfc8b1887416593d73f0003f38536e90fd285e0c8e415b30ea"}
Jul 30 13:01:42.134344 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.134277    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"]
Jul 30 13:01:42.146552 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.146527    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.149289 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.149260    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"]
Jul 30 13:01:42.149511 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.149491    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 30 13:01:42.149511 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.149499    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-437946spr3glq\""
Jul 30 13:01:42.149728 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.149516    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 30 13:01:42.149728 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.149572    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 30 13:01:42.149728 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.149587    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 30 13:01:42.149911 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.149742    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 30 13:01:42.149911 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.149747    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-xbvff\""
Jul 30 13:01:42.239136 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.239103    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-grpc-tls\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.239305 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.239157    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.239305 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.239179    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.239305 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.239282    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-tls\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.239431 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.239321    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.239431 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.239381    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.239431 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.239405    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpmt4\" (UniqueName: \"kubernetes.io/projected/dddb41df-4d00-4008-a164-6b21d1656174-kube-api-access-gpmt4\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.239548 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.239433    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dddb41df-4d00-4008-a164-6b21d1656174-metrics-client-ca\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.340480 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.340441    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-tls\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.340674 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.340489    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.340674 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.340524    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.340674 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.340547    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpmt4\" (UniqueName: \"kubernetes.io/projected/dddb41df-4d00-4008-a164-6b21d1656174-kube-api-access-gpmt4\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.340674 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.340570    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dddb41df-4d00-4008-a164-6b21d1656174-metrics-client-ca\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.340674 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.340664    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-grpc-tls\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.340947 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.340724    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.340947 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.340751    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.341558 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.341504    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dddb41df-4d00-4008-a164-6b21d1656174-metrics-client-ca\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.343488 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.343461    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.343712 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.343689    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-grpc-tls\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.343712 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.343702    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-tls\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.344160 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.344135    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.344427 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.344402    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.344643 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.344622    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dddb41df-4d00-4008-a164-6b21d1656174-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.350564 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.350540    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpmt4\" (UniqueName: \"kubernetes.io/projected/dddb41df-4d00-4008-a164-6b21d1656174-kube-api-access-gpmt4\") pod \"thanos-querier-6c6c58dd48-55hqk\" (UID: \"dddb41df-4d00-4008-a164-6b21d1656174\") " pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.457611 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.457555    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:42.596254 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.596153    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"]
Jul 30 13:01:42.702631 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:01:42.702561    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddddb41df_4d00_4008_a164_6b21d1656174.slice/crio-f182b7f3906fe8bdb3d75c36853d966bc1205a5268a5e4a18f9ce78e2b209dfc WatchSource:0}: Error finding container f182b7f3906fe8bdb3d75c36853d966bc1205a5268a5e4a18f9ce78e2b209dfc: Status 404 returned error can't find the container with id f182b7f3906fe8bdb3d75c36853d966bc1205a5268a5e4a18f9ce78e2b209dfc
Jul 30 13:01:42.736243 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:42.736220    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk" event={"ID":"dddb41df-4d00-4008-a164-6b21d1656174","Type":"ContainerStarted","Data":"f182b7f3906fe8bdb3d75c36853d966bc1205a5268a5e4a18f9ce78e2b209dfc"}
Jul 30 13:01:43.743025 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:43.742944    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv" event={"ID":"c136d17d-f123-49f0-aef4-990cfdea139c","Type":"ContainerStarted","Data":"608701309f52a7595e6a1c0905aa47a0f65d0b1d3d6f8c155f3f554999b2480d"}
Jul 30 13:01:43.744917 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:43.744890    2557 generic.go:356] "Generic (PLEG): container finished" podID="c9c400f7-8b1b-4394-8226-fc7e317de93a" containerID="41553c9e17204107c33c6c7ee1e38cc55f4a9ff6ab615401edb0141b2317bb81" exitCode=0
Jul 30 13:01:43.745050 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:43.744929    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7k62v" event={"ID":"c9c400f7-8b1b-4394-8226-fc7e317de93a","Type":"ContainerDied","Data":"41553c9e17204107c33c6c7ee1e38cc55f4a9ff6ab615401edb0141b2317bb81"}
Jul 30 13:01:43.761905 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:43.761861    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gg6kv" podStartSLOduration=1.90275975 podStartE2EDuration="3.76184464s" totalImagesPullingTime="1.85908489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:01:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:01:43.761464757 +0000 UTC m=+127.033929165" watchObservedRunningTime="2026-07-30 13:01:43.76184464 +0000 UTC m=+127.034309036"
Jul 30 13:01:44.465997 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.465959    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6c9cb979f-pdhpb"]
Jul 30 13:01:44.495256 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.495216    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c9cb979f-pdhpb"]
Jul 30 13:01:44.495424 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.495343    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.498272 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.498248    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 30 13:01:44.498417 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.498276    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 30 13:01:44.498417 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.498254    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 30 13:01:44.498529 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.498248    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 30 13:01:44.499637 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.499613    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3c4a4udvbefn4\""
Jul 30 13:01:44.499732 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.499620    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-lrrzh\""
Jul 30 13:01:44.560133 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.560095    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-secret-metrics-server-client-certs\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.560133 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.560134    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.560372 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.560207    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-metrics-server-audit-profiles\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.560372 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.560276    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-audit-log\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.560372 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.560299    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-secret-metrics-server-tls\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.560372 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.560335    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vrng\" (UniqueName: \"kubernetes.io/projected/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-kube-api-access-6vrng\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.560372 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.560358    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-client-ca-bundle\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.661564 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.661524    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-metrics-server-audit-profiles\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.661755 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.661609    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-audit-log\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.661755 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.661642    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-secret-metrics-server-tls\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.661755 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.661668    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vrng\" (UniqueName: \"kubernetes.io/projected/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-kube-api-access-6vrng\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.661755 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.661692    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-client-ca-bundle\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.662003 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.661757    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-secret-metrics-server-client-certs\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.662003 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.661785    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.662464 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.662346    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-audit-log\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.662464 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.662457    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.662754 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.662712    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-metrics-server-audit-profiles\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.664662 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.664635    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-client-ca-bundle\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.664800 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.664777    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-secret-metrics-server-client-certs\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.665204 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.665185    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-secret-metrics-server-tls\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.669982 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.669961    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vrng\" (UniqueName: \"kubernetes.io/projected/21f1dec4-01ad-4e73-a3c5-a5244bbcfddb-kube-api-access-6vrng\") pod \"metrics-server-6c9cb979f-pdhpb\" (UID: \"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb\") " pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.752001 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.751920    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7k62v" event={"ID":"c9c400f7-8b1b-4394-8226-fc7e317de93a","Type":"ContainerStarted","Data":"6e1d0f3e803788391ac685e8eeda147022157fac24043517881c9734c4e6dd2b"}
Jul 30 13:01:44.752001 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.751989    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7k62v" event={"ID":"c9c400f7-8b1b-4394-8226-fc7e317de93a","Type":"ContainerStarted","Data":"ca633c34d69d1a616cf06e93f5ab3eeb4980a1aec3fc3d1577ab4c03a2b702ce"}
Jul 30 13:01:44.774794 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.774703    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7k62v" podStartSLOduration=2.349407312 podStartE2EDuration="4.774684808s" totalImagesPullingTime="2.334074984s" totalInitContainerRuntime="91.202512ms" isStatefulPod=true podCreationTimestamp="2026-07-30 13:01:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:01:44.77396991 +0000 UTC m=+128.046434327" watchObservedRunningTime="2026-07-30 13:01:44.774684808 +0000 UTC m=+128.047149206"
Jul 30 13:01:44.805945 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.805919    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:01:44.834979 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.834941    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz"]
Jul 30 13:01:44.863970 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.863940    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz"]
Jul 30 13:01:44.864135 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.864048    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz"
Jul 30 13:01:44.866838 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.866812    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 30 13:01:44.866960 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.866900    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-r27zl\""
Jul 30 13:01:44.965678 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.965644    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnslz\" (UniqueName: \"kubernetes.io/projected/d57fc4c7-6da1-499b-9bd5-84450e3da020-kube-api-access-cnslz\") pod \"monitoring-plugin-68b7fc9574-xd6pz\" (UID: \"d57fc4c7-6da1-499b-9bd5-84450e3da020\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz"
Jul 30 13:01:44.965678 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:44.965678    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d57fc4c7-6da1-499b-9bd5-84450e3da020-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-xd6pz\" (UID: \"d57fc4c7-6da1-499b-9bd5-84450e3da020\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz"
Jul 30 13:01:45.066126 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:45.066097    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnslz\" (UniqueName: \"kubernetes.io/projected/d57fc4c7-6da1-499b-9bd5-84450e3da020-kube-api-access-cnslz\") pod \"monitoring-plugin-68b7fc9574-xd6pz\" (UID: \"d57fc4c7-6da1-499b-9bd5-84450e3da020\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz"
Jul 30 13:01:45.066126 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:45.066133    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d57fc4c7-6da1-499b-9bd5-84450e3da020-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-xd6pz\" (UID: \"d57fc4c7-6da1-499b-9bd5-84450e3da020\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz"
Jul 30 13:01:45.068481 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:45.068454    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d57fc4c7-6da1-499b-9bd5-84450e3da020-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-xd6pz\" (UID: \"d57fc4c7-6da1-499b-9bd5-84450e3da020\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz"
Jul 30 13:01:45.075878 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:45.075858    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnslz\" (UniqueName: \"kubernetes.io/projected/d57fc4c7-6da1-499b-9bd5-84450e3da020-kube-api-access-cnslz\") pod \"monitoring-plugin-68b7fc9574-xd6pz\" (UID: \"d57fc4c7-6da1-499b-9bd5-84450e3da020\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz"
Jul 30 13:01:45.128508 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:45.128406    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c9cb979f-pdhpb"]
Jul 30 13:01:45.130958 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:01:45.130934    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f1dec4_01ad_4e73_a3c5_a5244bbcfddb.slice/crio-00916dfc8cae46e7b22022bd86b801d291fd772334fa40b1128f991370ec4475 WatchSource:0}: Error finding container 00916dfc8cae46e7b22022bd86b801d291fd772334fa40b1128f991370ec4475: Status 404 returned error can't find the container with id 00916dfc8cae46e7b22022bd86b801d291fd772334fa40b1128f991370ec4475
Jul 30 13:01:45.175665 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:45.175643    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz"
Jul 30 13:01:45.496047 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:45.496022    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz"]
Jul 30 13:01:45.498849 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:01:45.498821    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd57fc4c7_6da1_499b_9bd5_84450e3da020.slice/crio-759bd44db9ce4faa229ca8d58d648badd22f83dc9d64515048a8813a15e2f7b5 WatchSource:0}: Error finding container 759bd44db9ce4faa229ca8d58d648badd22f83dc9d64515048a8813a15e2f7b5: Status 404 returned error can't find the container with id 759bd44db9ce4faa229ca8d58d648badd22f83dc9d64515048a8813a15e2f7b5
Jul 30 13:01:45.757069 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:45.757034    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk" event={"ID":"dddb41df-4d00-4008-a164-6b21d1656174","Type":"ContainerStarted","Data":"ce20817c73c093c07328ca5ca4b994b30e81b10b2292d1add013048a11c58e87"}
Jul 30 13:01:45.757506 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:45.757116    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk" event={"ID":"dddb41df-4d00-4008-a164-6b21d1656174","Type":"ContainerStarted","Data":"2a441e45e07e8d52abffddddd0ca5b68c402581a3997b8d56be49fbb51edff48"}
Jul 30 13:01:45.757506 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:45.757155    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk" event={"ID":"dddb41df-4d00-4008-a164-6b21d1656174","Type":"ContainerStarted","Data":"d83e0685dc0b38d50f608268d2c441317281d9524ede48e7aa429e6593db492e"}
Jul 30 13:01:45.758189 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:45.758161    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz" event={"ID":"d57fc4c7-6da1-499b-9bd5-84450e3da020","Type":"ContainerStarted","Data":"759bd44db9ce4faa229ca8d58d648badd22f83dc9d64515048a8813a15e2f7b5"}
Jul 30 13:01:45.759228 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:45.759205    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb" event={"ID":"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb","Type":"ContainerStarted","Data":"00916dfc8cae46e7b22022bd86b801d291fd772334fa40b1128f991370ec4475"}
Jul 30 13:01:45.875120 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:45.875090    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs\") pod \"network-metrics-daemon-74gmj\" (UID: \"c083abff-9168-456b-ba24-dd906dc138b2\") " pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 13:01:45.877483 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:45.877457    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c083abff-9168-456b-ba24-dd906dc138b2-metrics-certs\") pod \"network-metrics-daemon-74gmj\" (UID: \"c083abff-9168-456b-ba24-dd906dc138b2\") " pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 13:01:46.099364 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.099330    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v4mfl\""
Jul 30 13:01:46.107541 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.107504    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-74gmj"
Jul 30 13:01:46.273224 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.273194    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-74gmj"]
Jul 30 13:01:46.276097 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:01:46.276067    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc083abff_9168_456b_ba24_dd906dc138b2.slice/crio-8ea81c549b9b5f7f50b57ea8cbc26fcb83f56f070b2a5e7a8f923b29121ecb30 WatchSource:0}: Error finding container 8ea81c549b9b5f7f50b57ea8cbc26fcb83f56f070b2a5e7a8f923b29121ecb30: Status 404 returned error can't find the container with id 8ea81c549b9b5f7f50b57ea8cbc26fcb83f56f070b2a5e7a8f923b29121ecb30
Jul 30 13:01:46.320412 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.320374    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 13:01:46.330184 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.330156    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.333661 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.333634    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 30 13:01:46.333759 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.333710    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 30 13:01:46.333859 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.333844    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 30 13:01:46.334320 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.334153    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 30 13:01:46.334320 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.334176    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 30 13:01:46.334320 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.334203    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 30 13:01:46.334528 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.334358    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-76v8k\""
Jul 30 13:01:46.334528 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.334434    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 30 13:01:46.334842 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.334826    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4h3upol8vj65c\""
Jul 30 13:01:46.335077 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.335063    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 30 13:01:46.335967 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.335682    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 30 13:01:46.335967 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.335686    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 30 13:01:46.337351 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.337331    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 30 13:01:46.347321 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.347301    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 30 13:01:46.352335 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.352286    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 30 13:01:46.359974 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.359957    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 30 13:01:46.368166 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.368144    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 13:01:46.379758 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.379737    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.379855 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.379772    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.379855 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.379805    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.379963 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.379853    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/510e5683-2a6c-4b08-84d9-2680a2b3f824-config-out\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.379963 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.379905    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.379963 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.379936    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.380077 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.379974    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.380077 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.380033    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-config\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.380077 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.380070    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/510e5683-2a6c-4b08-84d9-2680a2b3f824-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.380205 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.380091    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.380205 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.380120    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.380205 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.380147    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.380205 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.380190    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.380400 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.380242    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.380400 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.380269    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s44j6\" (UniqueName: \"kubernetes.io/projected/510e5683-2a6c-4b08-84d9-2680a2b3f824-kube-api-access-s44j6\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.380400 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.380305    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.380400 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.380342    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.380400 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.380366    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.380400 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.380390    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-web-config\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.380684 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.380424    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.480934 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.480894    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481136 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.480945    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481136 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.480988    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481136 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481026    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481136 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481045    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/510e5683-2a6c-4b08-84d9-2680a2b3f824-config-out\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481136 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481091    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481136 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481117    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481472 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481166    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481472 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481201    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-config\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481472 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481225    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/510e5683-2a6c-4b08-84d9-2680a2b3f824-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481472 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481240    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481472 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481264    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481472 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481296    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481472 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481327    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481472 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481353    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481472 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481367    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s44j6\" (UniqueName: \"kubernetes.io/projected/510e5683-2a6c-4b08-84d9-2680a2b3f824-kube-api-access-s44j6\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481472 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481416    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.481472 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481459    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.482049 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481483    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.482049 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.481506    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-web-config\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.482213 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.482181    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.482927 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.482891    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.485646 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.485276    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.486052 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.485833    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.486529 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.486504    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.486652 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.486612    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.486729 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.486704    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.486970 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.486919    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-web-config\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.487144 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.487127    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/510e5683-2a6c-4b08-84d9-2680a2b3f824-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.488560 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.488094    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.488560 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.488247    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.488560 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.488259    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.488560 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.488313    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/510e5683-2a6c-4b08-84d9-2680a2b3f824-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.488560 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.488396    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.488560 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.488429    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.488955 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.488615    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-config\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.488955 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.488711    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.489519 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.489495    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/510e5683-2a6c-4b08-84d9-2680a2b3f824-config-out\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.489804 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.489780    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/510e5683-2a6c-4b08-84d9-2680a2b3f824-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.492134 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.492104    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s44j6\" (UniqueName: \"kubernetes.io/projected/510e5683-2a6c-4b08-84d9-2680a2b3f824-kube-api-access-s44j6\") pod \"prometheus-k8s-0\" (UID: \"510e5683-2a6c-4b08-84d9-2680a2b3f824\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.642473 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.642395    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:01:46.764649 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:46.764612    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-74gmj" event={"ID":"c083abff-9168-456b-ba24-dd906dc138b2","Type":"ContainerStarted","Data":"8ea81c549b9b5f7f50b57ea8cbc26fcb83f56f070b2a5e7a8f923b29121ecb30"}
Jul 30 13:01:47.118614 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:47.118569    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 13:01:47.122412 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:01:47.122363    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod510e5683_2a6c_4b08_84d9_2680a2b3f824.slice/crio-dee41bc88209bf6aca2e18702243dea068dc164c9bca730a492ae550e71263b4 WatchSource:0}: Error finding container dee41bc88209bf6aca2e18702243dea068dc164c9bca730a492ae550e71263b4: Status 404 returned error can't find the container with id dee41bc88209bf6aca2e18702243dea068dc164c9bca730a492ae550e71263b4
Jul 30 13:01:47.771988 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:47.771943    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb" event={"ID":"21f1dec4-01ad-4e73-a3c5-a5244bbcfddb","Type":"ContainerStarted","Data":"d2ce94df8d34f63d985761906228c53401ef4dde4a777b9cf99b3be1b4ceb840"}
Jul 30 13:01:47.773757 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:47.773681    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"510e5683-2a6c-4b08-84d9-2680a2b3f824","Type":"ContainerStarted","Data":"dee41bc88209bf6aca2e18702243dea068dc164c9bca730a492ae550e71263b4"}
Jul 30 13:01:47.777917 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:47.777883    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk" event={"ID":"dddb41df-4d00-4008-a164-6b21d1656174","Type":"ContainerStarted","Data":"d082a5163a79b0937673d9be9ded62ff561f5e4c12a7989f729500a268fe149c"}
Jul 30 13:01:47.778034 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:47.777957    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk" event={"ID":"dddb41df-4d00-4008-a164-6b21d1656174","Type":"ContainerStarted","Data":"3c08993c93e0a3c25077408dc2ea9996e4b6f04e6f000d25013fa54c4206d7da"}
Jul 30 13:01:47.778034 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:47.778005    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk" event={"ID":"dddb41df-4d00-4008-a164-6b21d1656174","Type":"ContainerStarted","Data":"8a5356614f2698ed9379f7f6c3ffb5bf6f59ed3454abcc2452bb4ec78910de17"}
Jul 30 13:01:47.778322 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:47.778251    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:47.793859 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:47.793736    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb" podStartSLOduration=1.9497258259999999 podStartE2EDuration="3.793715001s" totalImagesPullingTime="1.843989175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:01:47.792318124 +0000 UTC m=+131.064782523" watchObservedRunningTime="2026-07-30 13:01:47.793715001 +0000 UTC m=+131.066179416"
Jul 30 13:01:47.816726 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:47.816677    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk" podStartSLOduration=1.894313805 podStartE2EDuration="5.816660676s" totalImagesPullingTime="3.922346871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:01:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:01:47.81439879 +0000 UTC m=+131.086863212" watchObservedRunningTime="2026-07-30 13:01:47.816660676 +0000 UTC m=+131.089125072"
Jul 30 13:01:48.783246 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:48.783154    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-74gmj" event={"ID":"c083abff-9168-456b-ba24-dd906dc138b2","Type":"ContainerStarted","Data":"d934945a773b6030d257be554d4e4216efbcc4171766e89c7aaf8ce0a7daa1c5"}
Jul 30 13:01:48.783246 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:48.783224    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-74gmj" event={"ID":"c083abff-9168-456b-ba24-dd906dc138b2","Type":"ContainerStarted","Data":"090f76698c8600e0a771896a770d592185332830069f11aed5cdd8b039d7bdf8"}
Jul 30 13:01:48.784606 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:48.784566    2557 generic.go:356] "Generic (PLEG): container finished" podID="510e5683-2a6c-4b08-84d9-2680a2b3f824" containerID="983e394f2a195b47a9275c6c1e6ecef74485fed4c0b091b682dd2417c3be6433" exitCode=0
Jul 30 13:01:48.784711 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:48.784652    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"510e5683-2a6c-4b08-84d9-2680a2b3f824","Type":"ContainerDied","Data":"983e394f2a195b47a9275c6c1e6ecef74485fed4c0b091b682dd2417c3be6433"}
Jul 30 13:01:48.786202 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:48.786178    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz" event={"ID":"d57fc4c7-6da1-499b-9bd5-84450e3da020","Type":"ContainerStarted","Data":"9c8d1a3e5297e869ec8336eb90162afaa5140ef51a9dacc0cbc8dca656aa10f3"}
Jul 30 13:01:48.801259 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:48.801215    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-74gmj" podStartSLOduration=128.64200778 podStartE2EDuration="2m10.801201311s" totalImagesPullingTime="2.159193531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 12:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:01:48.800403691 +0000 UTC m=+132.072868112" watchObservedRunningTime="2026-07-30 13:01:48.801201311 +0000 UTC m=+132.073665709"
Jul 30 13:01:48.818152 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:48.818111    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz" podStartSLOduration=1.880351605 podStartE2EDuration="4.81809709s" totalImagesPullingTime="2.937745485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:01:48.816645413 +0000 UTC m=+132.089109829" watchObservedRunningTime="2026-07-30 13:01:48.81809709 +0000 UTC m=+132.090561487"
Jul 30 13:01:49.793391 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:49.793352    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz"
Jul 30 13:01:49.797485 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:49.797465    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xd6pz"
Jul 30 13:01:52.806811 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:52.806782    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"510e5683-2a6c-4b08-84d9-2680a2b3f824","Type":"ContainerStarted","Data":"1d758c28910e49f452d7f02b605f5cd0bd63a5aceee3a4738304134772c3d10a"}
Jul 30 13:01:52.807178 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:52.806863    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"510e5683-2a6c-4b08-84d9-2680a2b3f824","Type":"ContainerStarted","Data":"0eaaf9679df9e05081facca97be8da131bfc60696720bf9e805d21d1fce1b360"}
Jul 30 13:01:52.807178 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:52.806906    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"510e5683-2a6c-4b08-84d9-2680a2b3f824","Type":"ContainerStarted","Data":"a106c80145711d9cba9ee21472abaa35964f15a773b3ed023a8d32eb3a68358f"}
Jul 30 13:01:53.792875 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:53.792848    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6c6c58dd48-55hqk"
Jul 30 13:01:53.814416 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:53.814388    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"510e5683-2a6c-4b08-84d9-2680a2b3f824","Type":"ContainerStarted","Data":"d0a6f047f4082610f191877b430fe0938393ab2c1a0e39a8b4cf6377e6f79b65"}
Jul 30 13:01:53.814751 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:53.814447    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"510e5683-2a6c-4b08-84d9-2680a2b3f824","Type":"ContainerStarted","Data":"124f02e1417e4383a6c927e2bc2533186e13a4d7a5a732a5b53c688fc3ea383a"}
Jul 30 13:01:53.814751 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:53.814479    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"510e5683-2a6c-4b08-84d9-2680a2b3f824","Type":"ContainerStarted","Data":"496990b0ef49ca94ff40eb16ee753f638db90136892ae7188d013e1705575955"}
Jul 30 13:01:53.849444 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:53.849353    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.762865106 podStartE2EDuration="7.849331021s" totalImagesPullingTime="5.06555033s" totalInitContainerRuntime="20.915585ms" isStatefulPod=false podCreationTimestamp="2026-07-30 13:01:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:01:53.847439167 +0000 UTC m=+137.119903588" watchObservedRunningTime="2026-07-30 13:01:53.849331021 +0000 UTC m=+137.121795419"
Jul 30 13:01:56.642999 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:01:56.642958    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:02:04.806749 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:02:04.806704    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:02:04.807133 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:02:04.806775    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:02:14.896192 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:02:14.896112    2557 generic.go:356] "Generic (PLEG): container finished" podID="5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0" containerID="b446dd8e4046e51dac9ed87d22e736cc63d30cff0185d9a9d0675de6efdb8ffb" exitCode=0
Jul 30 13:02:14.896192 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:02:14.896184    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-pr87k" event={"ID":"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0","Type":"ContainerDied","Data":"b446dd8e4046e51dac9ed87d22e736cc63d30cff0185d9a9d0675de6efdb8ffb"}
Jul 30 13:02:14.896581 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:02:14.896494    2557 scope.go:117] "RemoveContainer" containerID="b446dd8e4046e51dac9ed87d22e736cc63d30cff0185d9a9d0675de6efdb8ffb"
Jul 30 13:02:15.902111 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:02:15.902081    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-pr87k" event={"ID":"5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0","Type":"ContainerStarted","Data":"bbd96206d4f893210063e69e3953b9ea1025997cedcd907879cec1bb715e86ef"}
Jul 30 13:02:24.811915 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:02:24.811883    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:02:24.815511 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:02:24.815489    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6c9cb979f-pdhpb"
Jul 30 13:02:46.642749 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:02:46.642712    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:02:46.661446 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:02:46.661416    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:02:47.034907 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:02:47.034879    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 13:04:37.081591 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:04:37.081563    2557 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 13:05:03.670656 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.670620    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"]
Jul 30 13:05:03.674146 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.674129    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.678248 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.678225    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 13:05:03.678872 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.678850    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 13:05:03.678992 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.678928    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 13:05:03.678992 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.678981    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 13:05:03.680628 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.680430    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cwt6z\""
Jul 30 13:05:03.694667 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.694636    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"]
Jul 30 13:05:03.772537 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.772487    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-manifests\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.772537 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.772544    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-tmp\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.772782 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.772569    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.772782 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.772632    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l89l\" (UniqueName: \"kubernetes.io/projected/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-kube-api-access-6l89l\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.772782 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.772744    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.873677 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.873641    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.873677 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.873683    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-manifests\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.873885 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.873713    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-tmp\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.873885 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.873740    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.873885 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.873771    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l89l\" (UniqueName: \"kubernetes.io/projected/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-kube-api-access-6l89l\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.874122 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.874101    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-manifests\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.876107 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.876078    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-tmp\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.876306 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.876287    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.876351 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.876315    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.884953 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.884932    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l89l\" (UniqueName: \"kubernetes.io/projected/d9cda7fe-e008-4ffe-9f50-0a87e31e53ce-kube-api-access-6l89l\") pod \"opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6\" (UID: \"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:03.984287 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:03.984185    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:04.143230 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:04.143147    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"]
Jul 30 13:05:04.145732 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:05:04.145704    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9cda7fe_e008_4ffe_9f50_0a87e31e53ce.slice/crio-c7bfe109993631da0db096aad79982bd6d74dc67d6e6a8cb59c0346031db6457 WatchSource:0}: Error finding container c7bfe109993631da0db096aad79982bd6d74dc67d6e6a8cb59c0346031db6457: Status 404 returned error can't find the container with id c7bfe109993631da0db096aad79982bd6d74dc67d6e6a8cb59c0346031db6457
Jul 30 13:05:04.147486 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:04.147469    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 13:05:04.526914 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:04.526879    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6" event={"ID":"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce","Type":"ContainerStarted","Data":"c7bfe109993631da0db096aad79982bd6d74dc67d6e6a8cb59c0346031db6457"}
Jul 30 13:05:07.541521 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:07.541479    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6" event={"ID":"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce","Type":"ContainerStarted","Data":"443b5d1e6fa66bce5278f495572829c4e2a014a308136cba6a7a82def62a18b5"}
Jul 30 13:05:08.546784 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:08.546753    2557 generic.go:356] "Generic (PLEG): container finished" podID="d9cda7fe-e008-4ffe-9f50-0a87e31e53ce" containerID="443b5d1e6fa66bce5278f495572829c4e2a014a308136cba6a7a82def62a18b5" exitCode=0
Jul 30 13:05:08.547139 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:08.546845    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6" event={"ID":"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce","Type":"ContainerDied","Data":"443b5d1e6fa66bce5278f495572829c4e2a014a308136cba6a7a82def62a18b5"}
Jul 30 13:05:09.553192 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:09.553151    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6" event={"ID":"d9cda7fe-e008-4ffe-9f50-0a87e31e53ce","Type":"ContainerStarted","Data":"2cd0ebca3fc2d6a446b06a78a4246bd6e4714c715e82a8689c66fe1f928811fb"}
Jul 30 13:05:09.553574 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:09.553251    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:09.594861 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:09.594802    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6" podStartSLOduration=1.943239377 podStartE2EDuration="6.594783689s" totalImagesPullingTime="3.59069324s" totalInitContainerRuntime="1.060851072s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:05:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:05:09.593261318 +0000 UTC m=+332.865725717" watchObservedRunningTime="2026-07-30 13:05:09.594783689 +0000 UTC m=+332.867248086"
Jul 30 13:05:20.559535 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:20.559501    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6"
Jul 30 13:05:21.210469 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.210438    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"]
Jul 30 13:05:21.219254 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.219233    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:21.223910 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.223887    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 30 13:05:21.224143 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.223924    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 30 13:05:21.224231 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.224162    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 30 13:05:21.224231 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.223940    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5xxkh\""
Jul 30 13:05:21.224339 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.223995    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 30 13:05:21.225086 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.224978    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 30 13:05:21.226081 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.226058    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"]
Jul 30 13:05:21.332502 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.332470    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/964335af-e13d-4f77-9a05-0e58dc53c637-metrics-cert\") pod \"lws-controller-manager-d8f5f8447-zhn9c\" (UID: \"964335af-e13d-4f77-9a05-0e58dc53c637\") " pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:21.332502 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.332500    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9jw2\" (UniqueName: \"kubernetes.io/projected/964335af-e13d-4f77-9a05-0e58dc53c637-kube-api-access-n9jw2\") pod \"lws-controller-manager-d8f5f8447-zhn9c\" (UID: \"964335af-e13d-4f77-9a05-0e58dc53c637\") " pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:21.332716 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.332526    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/964335af-e13d-4f77-9a05-0e58dc53c637-cert\") pod \"lws-controller-manager-d8f5f8447-zhn9c\" (UID: \"964335af-e13d-4f77-9a05-0e58dc53c637\") " pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:21.332716 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.332698    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/964335af-e13d-4f77-9a05-0e58dc53c637-manager-config\") pod \"lws-controller-manager-d8f5f8447-zhn9c\" (UID: \"964335af-e13d-4f77-9a05-0e58dc53c637\") " pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:21.433792 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.433756    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/964335af-e13d-4f77-9a05-0e58dc53c637-manager-config\") pod \"lws-controller-manager-d8f5f8447-zhn9c\" (UID: \"964335af-e13d-4f77-9a05-0e58dc53c637\") " pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:21.433792 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.433798    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/964335af-e13d-4f77-9a05-0e58dc53c637-metrics-cert\") pod \"lws-controller-manager-d8f5f8447-zhn9c\" (UID: \"964335af-e13d-4f77-9a05-0e58dc53c637\") " pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:21.433992 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.433820    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9jw2\" (UniqueName: \"kubernetes.io/projected/964335af-e13d-4f77-9a05-0e58dc53c637-kube-api-access-n9jw2\") pod \"lws-controller-manager-d8f5f8447-zhn9c\" (UID: \"964335af-e13d-4f77-9a05-0e58dc53c637\") " pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:21.433992 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.433853    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/964335af-e13d-4f77-9a05-0e58dc53c637-cert\") pod \"lws-controller-manager-d8f5f8447-zhn9c\" (UID: \"964335af-e13d-4f77-9a05-0e58dc53c637\") " pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:21.434411 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.434391    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/964335af-e13d-4f77-9a05-0e58dc53c637-manager-config\") pod \"lws-controller-manager-d8f5f8447-zhn9c\" (UID: \"964335af-e13d-4f77-9a05-0e58dc53c637\") " pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:21.436321 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.436304    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/964335af-e13d-4f77-9a05-0e58dc53c637-metrics-cert\") pod \"lws-controller-manager-d8f5f8447-zhn9c\" (UID: \"964335af-e13d-4f77-9a05-0e58dc53c637\") " pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:21.436392 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.436360    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/964335af-e13d-4f77-9a05-0e58dc53c637-cert\") pod \"lws-controller-manager-d8f5f8447-zhn9c\" (UID: \"964335af-e13d-4f77-9a05-0e58dc53c637\") " pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:21.442502 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.442465    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9jw2\" (UniqueName: \"kubernetes.io/projected/964335af-e13d-4f77-9a05-0e58dc53c637-kube-api-access-n9jw2\") pod \"lws-controller-manager-d8f5f8447-zhn9c\" (UID: \"964335af-e13d-4f77-9a05-0e58dc53c637\") " pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:21.530023 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.529954    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:21.656082 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:21.656050    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"]
Jul 30 13:05:21.657335 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:05:21.657305    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod964335af_e13d_4f77_9a05_0e58dc53c637.slice/crio-9a7b7284579beaf580a4fb9d331aceb07924e2a9a87d3f9e72926f70cc9b6c62 WatchSource:0}: Error finding container 9a7b7284579beaf580a4fb9d331aceb07924e2a9a87d3f9e72926f70cc9b6c62: Status 404 returned error can't find the container with id 9a7b7284579beaf580a4fb9d331aceb07924e2a9a87d3f9e72926f70cc9b6c62
Jul 30 13:05:22.613422 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:22.613386    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c" event={"ID":"964335af-e13d-4f77-9a05-0e58dc53c637","Type":"ContainerStarted","Data":"9a7b7284579beaf580a4fb9d331aceb07924e2a9a87d3f9e72926f70cc9b6c62"}
Jul 30 13:05:24.624330 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:24.624292    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c" event={"ID":"964335af-e13d-4f77-9a05-0e58dc53c637","Type":"ContainerStarted","Data":"91c4aaf983a85de5e143997b7c49e51d8dafef9240a36193c519e02c3adf5a66"}
Jul 30 13:05:24.624754 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:24.624404    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:05:24.654661 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:24.654587    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c" podStartSLOduration=1.192705973 podStartE2EDuration="3.654568404s" totalImagesPullingTime="2.461862431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:05:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:05:24.653658913 +0000 UTC m=+347.926123339" watchObservedRunningTime="2026-07-30 13:05:24.654568404 +0000 UTC m=+347.927032802"
Jul 30 13:05:25.379069 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.379032    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk"]
Jul 30 13:05:25.382836 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.382816    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk"
Jul 30 13:05:25.386258 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.386229    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 30 13:05:25.386525 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.386508    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 30 13:05:25.386574 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.386539    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tfg9p\""
Jul 30 13:05:25.388255 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.388237    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 13:05:25.388255 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.388239    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 13:05:25.401611 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.401565    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk"]
Jul 30 13:05:25.571832 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.571798    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a743ebf-e046-43c5-8e66-ef50e90ce5b5-tls-certs\") pod \"kube-auth-proxy-54d6c599cc-b8lfk\" (UID: \"4a743ebf-e046-43c5-8e66-ef50e90ce5b5\") " pod="openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk"
Jul 30 13:05:25.571983 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.571850    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a743ebf-e046-43c5-8e66-ef50e90ce5b5-tmp\") pod \"kube-auth-proxy-54d6c599cc-b8lfk\" (UID: \"4a743ebf-e046-43c5-8e66-ef50e90ce5b5\") " pod="openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk"
Jul 30 13:05:25.571983 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.571894    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nqhh\" (UniqueName: \"kubernetes.io/projected/4a743ebf-e046-43c5-8e66-ef50e90ce5b5-kube-api-access-4nqhh\") pod \"kube-auth-proxy-54d6c599cc-b8lfk\" (UID: \"4a743ebf-e046-43c5-8e66-ef50e90ce5b5\") " pod="openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk"
Jul 30 13:05:25.673121 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.673074    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a743ebf-e046-43c5-8e66-ef50e90ce5b5-tmp\") pod \"kube-auth-proxy-54d6c599cc-b8lfk\" (UID: \"4a743ebf-e046-43c5-8e66-ef50e90ce5b5\") " pod="openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk"
Jul 30 13:05:25.673750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.673127    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nqhh\" (UniqueName: \"kubernetes.io/projected/4a743ebf-e046-43c5-8e66-ef50e90ce5b5-kube-api-access-4nqhh\") pod \"kube-auth-proxy-54d6c599cc-b8lfk\" (UID: \"4a743ebf-e046-43c5-8e66-ef50e90ce5b5\") " pod="openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk"
Jul 30 13:05:25.673750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.673213    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a743ebf-e046-43c5-8e66-ef50e90ce5b5-tls-certs\") pod \"kube-auth-proxy-54d6c599cc-b8lfk\" (UID: \"4a743ebf-e046-43c5-8e66-ef50e90ce5b5\") " pod="openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk"
Jul 30 13:05:25.675492 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.675456    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a743ebf-e046-43c5-8e66-ef50e90ce5b5-tmp\") pod \"kube-auth-proxy-54d6c599cc-b8lfk\" (UID: \"4a743ebf-e046-43c5-8e66-ef50e90ce5b5\") " pod="openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk"
Jul 30 13:05:25.675792 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.675773    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a743ebf-e046-43c5-8e66-ef50e90ce5b5-tls-certs\") pod \"kube-auth-proxy-54d6c599cc-b8lfk\" (UID: \"4a743ebf-e046-43c5-8e66-ef50e90ce5b5\") " pod="openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk"
Jul 30 13:05:25.683310 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.683284    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nqhh\" (UniqueName: \"kubernetes.io/projected/4a743ebf-e046-43c5-8e66-ef50e90ce5b5-kube-api-access-4nqhh\") pod \"kube-auth-proxy-54d6c599cc-b8lfk\" (UID: \"4a743ebf-e046-43c5-8e66-ef50e90ce5b5\") " pod="openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk"
Jul 30 13:05:25.692962 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.692941    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk"
Jul 30 13:05:25.827758 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:25.827727    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk"]
Jul 30 13:05:25.830620 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:05:25.830570    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a743ebf_e046_43c5_8e66_ef50e90ce5b5.slice/crio-b267fc5827776dd103409fda87362532a65c94e6552d74591c1ef2173c1c0b80 WatchSource:0}: Error finding container b267fc5827776dd103409fda87362532a65c94e6552d74591c1ef2173c1c0b80: Status 404 returned error can't find the container with id b267fc5827776dd103409fda87362532a65c94e6552d74591c1ef2173c1c0b80
Jul 30 13:05:26.632756 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:26.632718    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk" event={"ID":"4a743ebf-e046-43c5-8e66-ef50e90ce5b5","Type":"ContainerStarted","Data":"b267fc5827776dd103409fda87362532a65c94e6552d74591c1ef2173c1c0b80"}
Jul 30 13:05:30.652497 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:30.652458    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk" event={"ID":"4a743ebf-e046-43c5-8e66-ef50e90ce5b5","Type":"ContainerStarted","Data":"856c0e8eea8e9a5a5f93f375b73db51b344e59f789fe23daa9aa97ea359c02ae"}
Jul 30 13:05:30.672220 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:30.672165    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-54d6c599cc-b8lfk" podStartSLOduration=1.697186139 podStartE2EDuration="5.67214412s" totalImagesPullingTime="3.974957981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:05:30.669483631 +0000 UTC m=+353.941948055" watchObservedRunningTime="2026-07-30 13:05:30.67214412 +0000 UTC m=+353.944608519"
Jul 30 13:05:35.630924 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:05:35.630884    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d8f5f8447-zhn9c"
Jul 30 13:07:11.904224 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:11.904145    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l"]
Jul 30 13:07:11.907415 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:11.907398    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l"
Jul 30 13:07:11.915108 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:11.915089    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 30 13:07:11.915205 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:11.915192    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 30 13:07:11.915880 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:11.915868    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 13:07:11.915940 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:11.915905    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 13:07:11.916643 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:11.916629    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-q85h5\""
Jul 30 13:07:11.922389 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:11.922360    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l"]
Jul 30 13:07:11.997823 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:11.997785    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/07468d65-d747-4283-8535-29d78f872f68-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5t68l\" (UID: \"07468d65-d747-4283-8535-29d78f872f68\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l"
Jul 30 13:07:11.997997 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:11.997844    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/07468d65-d747-4283-8535-29d78f872f68-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5t68l\" (UID: \"07468d65-d747-4283-8535-29d78f872f68\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l"
Jul 30 13:07:11.997997 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:11.997914    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwk65\" (UniqueName: \"kubernetes.io/projected/07468d65-d747-4283-8535-29d78f872f68-kube-api-access-bwk65\") pod \"kuadrant-console-plugin-66b5774985-5t68l\" (UID: \"07468d65-d747-4283-8535-29d78f872f68\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l"
Jul 30 13:07:12.098763 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:12.098728    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwk65\" (UniqueName: \"kubernetes.io/projected/07468d65-d747-4283-8535-29d78f872f68-kube-api-access-bwk65\") pod \"kuadrant-console-plugin-66b5774985-5t68l\" (UID: \"07468d65-d747-4283-8535-29d78f872f68\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l"
Jul 30 13:07:12.098950 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:12.098794    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/07468d65-d747-4283-8535-29d78f872f68-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5t68l\" (UID: \"07468d65-d747-4283-8535-29d78f872f68\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l"
Jul 30 13:07:12.098950 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:12.098847    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/07468d65-d747-4283-8535-29d78f872f68-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5t68l\" (UID: \"07468d65-d747-4283-8535-29d78f872f68\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l"
Jul 30 13:07:12.099457 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:12.099434    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/07468d65-d747-4283-8535-29d78f872f68-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5t68l\" (UID: \"07468d65-d747-4283-8535-29d78f872f68\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l"
Jul 30 13:07:12.101168 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:12.101145    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/07468d65-d747-4283-8535-29d78f872f68-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5t68l\" (UID: \"07468d65-d747-4283-8535-29d78f872f68\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l"
Jul 30 13:07:12.107636 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:12.107613    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwk65\" (UniqueName: \"kubernetes.io/projected/07468d65-d747-4283-8535-29d78f872f68-kube-api-access-bwk65\") pod \"kuadrant-console-plugin-66b5774985-5t68l\" (UID: \"07468d65-d747-4283-8535-29d78f872f68\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l"
Jul 30 13:07:12.216480 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:12.216451    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l"
Jul 30 13:07:12.339704 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:12.339681    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l"]
Jul 30 13:07:12.341894 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:07:12.341867    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07468d65_d747_4283_8535_29d78f872f68.slice/crio-43228cf8e8cf056486d2eb67339fbfeb913d5dba52185eaa071396e2e68f4883 WatchSource:0}: Error finding container 43228cf8e8cf056486d2eb67339fbfeb913d5dba52185eaa071396e2e68f4883: Status 404 returned error can't find the container with id 43228cf8e8cf056486d2eb67339fbfeb913d5dba52185eaa071396e2e68f4883
Jul 30 13:07:13.087741 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:13.087699    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l" event={"ID":"07468d65-d747-4283-8535-29d78f872f68","Type":"ContainerStarted","Data":"43228cf8e8cf056486d2eb67339fbfeb913d5dba52185eaa071396e2e68f4883"}
Jul 30 13:07:33.190653 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:33.190615    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l" event={"ID":"07468d65-d747-4283-8535-29d78f872f68","Type":"ContainerStarted","Data":"cb7e6a17514c64637618ace5d72247fd9d84f67df972e587254c9b9bad0619b5"}
Jul 30 13:07:33.208103 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:33.208058    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5t68l" podStartSLOduration=1.457155355 podStartE2EDuration="22.208042473s" totalImagesPullingTime="20.750887118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:07:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:07:33.206477521 +0000 UTC m=+476.478941915" watchObservedRunningTime="2026-07-30 13:07:33.208042473 +0000 UTC m=+476.480506870"
Jul 30 13:07:54.699458 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:54.699418    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:07:54.806175 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:54.806134    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:07:54.806175 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:54.806182    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:07:54.806380 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:54.806269    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"
Jul 30 13:07:54.809566 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:54.809540    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 13:07:54.883879 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:54.883837    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmjrc\" (UniqueName: \"kubernetes.io/projected/98e36e50-caba-4e26-a2b3-082265e732ac-kube-api-access-gmjrc\") pod \"limitador-limitador-54d6d988d5-bcmbj\" (UID: \"98e36e50-caba-4e26-a2b3-082265e732ac\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"
Jul 30 13:07:54.884062 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:54.883887    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98e36e50-caba-4e26-a2b3-082265e732ac-config-file\") pod \"limitador-limitador-54d6d988d5-bcmbj\" (UID: \"98e36e50-caba-4e26-a2b3-082265e732ac\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"
Jul 30 13:07:54.984810 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:54.984721    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmjrc\" (UniqueName: \"kubernetes.io/projected/98e36e50-caba-4e26-a2b3-082265e732ac-kube-api-access-gmjrc\") pod \"limitador-limitador-54d6d988d5-bcmbj\" (UID: \"98e36e50-caba-4e26-a2b3-082265e732ac\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"
Jul 30 13:07:54.984810 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:54.984760    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98e36e50-caba-4e26-a2b3-082265e732ac-config-file\") pod \"limitador-limitador-54d6d988d5-bcmbj\" (UID: \"98e36e50-caba-4e26-a2b3-082265e732ac\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"
Jul 30 13:07:54.985430 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:54.985411    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98e36e50-caba-4e26-a2b3-082265e732ac-config-file\") pod \"limitador-limitador-54d6d988d5-bcmbj\" (UID: \"98e36e50-caba-4e26-a2b3-082265e732ac\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"
Jul 30 13:07:54.994561 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:54.994532    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmjrc\" (UniqueName: \"kubernetes.io/projected/98e36e50-caba-4e26-a2b3-082265e732ac-kube-api-access-gmjrc\") pod \"limitador-limitador-54d6d988d5-bcmbj\" (UID: \"98e36e50-caba-4e26-a2b3-082265e732ac\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"
Jul 30 13:07:55.117416 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:55.117385    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"
Jul 30 13:07:55.251969 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:55.251944    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:07:55.254529 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:07:55.254504    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98e36e50_caba_4e26_a2b3_082265e732ac.slice/crio-a43894317c166a88ba34a8f46d67879e01a446bc3fd4143b97f92b1ed0d4e16d WatchSource:0}: Error finding container a43894317c166a88ba34a8f46d67879e01a446bc3fd4143b97f92b1ed0d4e16d: Status 404 returned error can't find the container with id a43894317c166a88ba34a8f46d67879e01a446bc3fd4143b97f92b1ed0d4e16d
Jul 30 13:07:55.280580 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:07:55.280553    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-bcmbj" event={"ID":"98e36e50-caba-4e26-a2b3-082265e732ac","Type":"ContainerStarted","Data":"a43894317c166a88ba34a8f46d67879e01a446bc3fd4143b97f92b1ed0d4e16d"}
Jul 30 13:08:00.303689 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:08:00.303648    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-bcmbj" event={"ID":"98e36e50-caba-4e26-a2b3-082265e732ac","Type":"ContainerStarted","Data":"8e83fbeed2b2b89c748d5d553db55c5361b74f76fd84580a8f083b3747f42a6f"}
Jul 30 13:08:00.304182 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:08:00.303838    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"
Jul 30 13:08:00.321466 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:08:00.321420    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-bcmbj" podStartSLOduration=1.409865593 podStartE2EDuration="6.32140469s" totalImagesPullingTime="4.911539097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:08:00.319796039 +0000 UTC m=+503.592260454" watchObservedRunningTime="2026-07-30 13:08:00.32140469 +0000 UTC m=+503.593869088"
Jul 30 13:08:11.309206 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:08:11.309175    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"
Jul 30 13:09:13.066326 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:09:13.066287    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:09:32.093612 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:09:32.093564    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:09:39.334770 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:09:39.334735    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:10:01.232799 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:10:01.232767    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:10:05.446101 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:10:05.446054    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:10:12.249806 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:10:12.249766    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:10:21.839806 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:10:21.839766    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:12:15.248223 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:12:15.248184    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:12:26.933711 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:12:26.933674    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:12:37.041097 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:12:37.041065    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:13:53.488558 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.488523    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b8b49f4d5-rzf74"]
Jul 30 13:13:53.491858 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.491840    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b8b49f4d5-rzf74"
Jul 30 13:13:53.494703 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.494675    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 13:13:53.494902 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.494889    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-c66mt\""
Jul 30 13:13:53.494969 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.494950    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 13:13:53.505634 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.505588    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b8b49f4d5-rzf74"]
Jul 30 13:13:53.598492 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.598462    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ddbc56d-98ea-4947-ad78-478d54248d40-webhook-cert\") pod \"maas-controller-b8b49f4d5-rzf74\" (UID: \"6ddbc56d-98ea-4947-ad78-478d54248d40\") " pod="opendatahub/maas-controller-b8b49f4d5-rzf74"
Jul 30 13:13:53.598668 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.598539    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj66r\" (UniqueName: \"kubernetes.io/projected/6ddbc56d-98ea-4947-ad78-478d54248d40-kube-api-access-sj66r\") pod \"maas-controller-b8b49f4d5-rzf74\" (UID: \"6ddbc56d-98ea-4947-ad78-478d54248d40\") " pod="opendatahub/maas-controller-b8b49f4d5-rzf74"
Jul 30 13:13:53.699569 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.699539    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj66r\" (UniqueName: \"kubernetes.io/projected/6ddbc56d-98ea-4947-ad78-478d54248d40-kube-api-access-sj66r\") pod \"maas-controller-b8b49f4d5-rzf74\" (UID: \"6ddbc56d-98ea-4947-ad78-478d54248d40\") " pod="opendatahub/maas-controller-b8b49f4d5-rzf74"
Jul 30 13:13:53.699756 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.699612    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ddbc56d-98ea-4947-ad78-478d54248d40-webhook-cert\") pod \"maas-controller-b8b49f4d5-rzf74\" (UID: \"6ddbc56d-98ea-4947-ad78-478d54248d40\") " pod="opendatahub/maas-controller-b8b49f4d5-rzf74"
Jul 30 13:13:53.702083 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.702061    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ddbc56d-98ea-4947-ad78-478d54248d40-webhook-cert\") pod \"maas-controller-b8b49f4d5-rzf74\" (UID: \"6ddbc56d-98ea-4947-ad78-478d54248d40\") " pod="opendatahub/maas-controller-b8b49f4d5-rzf74"
Jul 30 13:13:53.727068 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.727034    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj66r\" (UniqueName: \"kubernetes.io/projected/6ddbc56d-98ea-4947-ad78-478d54248d40-kube-api-access-sj66r\") pod \"maas-controller-b8b49f4d5-rzf74\" (UID: \"6ddbc56d-98ea-4947-ad78-478d54248d40\") " pod="opendatahub/maas-controller-b8b49f4d5-rzf74"
Jul 30 13:13:53.801687 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.801585    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b8b49f4d5-rzf74"
Jul 30 13:13:53.930361 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.930338    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b8b49f4d5-rzf74"]
Jul 30 13:13:53.933070 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:13:53.933032    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ddbc56d_98ea_4947_ad78_478d54248d40.slice/crio-816dccd425cb0170728f2b19a05f2cc0bd0fb3954a4913890ad040f4413a0246 WatchSource:0}: Error finding container 816dccd425cb0170728f2b19a05f2cc0bd0fb3954a4913890ad040f4413a0246: Status 404 returned error can't find the container with id 816dccd425cb0170728f2b19a05f2cc0bd0fb3954a4913890ad040f4413a0246
Jul 30 13:13:53.934882 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:53.934865    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 13:13:54.805930 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:54.805892    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b8b49f4d5-rzf74" event={"ID":"6ddbc56d-98ea-4947-ad78-478d54248d40","Type":"ContainerStarted","Data":"816dccd425cb0170728f2b19a05f2cc0bd0fb3954a4913890ad040f4413a0246"}
Jul 30 13:13:56.816259 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:56.816222    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b8b49f4d5-rzf74" event={"ID":"6ddbc56d-98ea-4947-ad78-478d54248d40","Type":"ContainerStarted","Data":"a489874293a042b7280d7092c75a4f1f8508aa05ddb46f5960b707d370e5d1f0"}
Jul 30 13:13:56.816692 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:56.816363    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b8b49f4d5-rzf74"
Jul 30 13:13:56.840796 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:13:56.837737    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b8b49f4d5-rzf74" podStartSLOduration=1.733169609 podStartE2EDuration="3.83771316s" totalImagesPullingTime="2.104543551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:13:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:13:56.834847918 +0000 UTC m=+860.107312316" watchObservedRunningTime="2026-07-30 13:13:56.83771316 +0000 UTC m=+860.110177561"
Jul 30 13:14:07.822678 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:14:07.822646    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b8b49f4d5-rzf74"
Jul 30 13:14:20.336353 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:14:20.336318    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:14:28.846197 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:14:28.846112    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:15:00.148796 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:00.148761    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756955-6wf9q"]
Jul 30 13:15:00.156198 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:00.156162    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756955-6wf9q"
Jul 30 13:15:00.159408 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:00.159384    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxdpz\""
Jul 30 13:15:00.159703 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:00.159682    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756955-6wf9q"]
Jul 30 13:15:00.167076 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:00.167059    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756955-6wf9q"
Jul 30 13:15:00.297312 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:00.297286    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756955-6wf9q"]
Jul 30 13:15:00.299835 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:15:00.299811    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f6d126_b213_4649_95c8_65727d0695cd.slice/crio-bb76c52b6edaa49cf0902e168248d0c9dcfbf9b6cbae9b46680e7aa8d60e131f WatchSource:0}: Error finding container bb76c52b6edaa49cf0902e168248d0c9dcfbf9b6cbae9b46680e7aa8d60e131f: Status 404 returned error can't find the container with id bb76c52b6edaa49cf0902e168248d0c9dcfbf9b6cbae9b46680e7aa8d60e131f
Jul 30 13:15:01.093080 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:01.093040    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756955-6wf9q" event={"ID":"68f6d126-b213-4649-95c8-65727d0695cd","Type":"ContainerStarted","Data":"bb76c52b6edaa49cf0902e168248d0c9dcfbf9b6cbae9b46680e7aa8d60e131f"}
Jul 30 13:15:03.103564 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:03.103527    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756955-6wf9q" event={"ID":"68f6d126-b213-4649-95c8-65727d0695cd","Type":"ContainerStarted","Data":"47fcd3c4f9424317e726ca9a59730792da61da1480c3e79543e3fdd5bedb0571"}
Jul 30 13:15:03.120415 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:03.120356    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756955-6wf9q" podStartSLOduration=0.607890397 podStartE2EDuration="3.120339803s" totalImagesPullingTime="2.512449406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:15:03.119858873 +0000 UTC m=+926.392323271" watchObservedRunningTime="2026-07-30 13:15:03.120339803 +0000 UTC m=+926.392804276"
Jul 30 13:15:04.109384 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:04.109294    2557 generic.go:356] "Generic (PLEG): container finished" podID="68f6d126-b213-4649-95c8-65727d0695cd" containerID="47fcd3c4f9424317e726ca9a59730792da61da1480c3e79543e3fdd5bedb0571" exitCode=0
Jul 30 13:15:04.109773 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:04.109377    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756955-6wf9q" event={"ID":"68f6d126-b213-4649-95c8-65727d0695cd","Type":"ContainerDied","Data":"47fcd3c4f9424317e726ca9a59730792da61da1480c3e79543e3fdd5bedb0571"}
Jul 30 13:15:05.241785 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:05.241760    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756955-6wf9q"
Jul 30 13:15:06.118979 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:06.118950    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756955-6wf9q"
Jul 30 13:15:06.120994 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:06.118949    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756955-6wf9q" event={"ID":"68f6d126-b213-4649-95c8-65727d0695cd","Type":"ContainerDied","Data":"bb76c52b6edaa49cf0902e168248d0c9dcfbf9b6cbae9b46680e7aa8d60e131f"}
Jul 30 13:15:06.120994 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:06.119116    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb76c52b6edaa49cf0902e168248d0c9dcfbf9b6cbae9b46680e7aa8d60e131f"
Jul 30 13:15:40.035531 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:40.035495    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:15:49.234562 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:15:49.234527    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:16:15.631919 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:16:15.631840    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:16:24.935270 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:16:24.935236    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:16:33.648986 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:16:33.648936    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:16:43.641283 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:16:43.641246    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:16:56.529905 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:16:56.529872    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:17:07.239631 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:17:07.239583    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:17:27.742674 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:17:27.742642    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:17:38.932905 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:17:38.932818    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:18:01.863366 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:18:01.863336    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:18:12.938944 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:18:12.938902    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:18:29.942237 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:18:29.942196    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:18:48.840093 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:18:48.840062    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:19:03.936702 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:19:03.936621    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:19:13.448474 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:19:13.448438    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:19:25.258932 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:19:25.258899    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:19:37.235821 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:19:37.235788    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:19:49.230867 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:19:49.230834    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:20:03.533062 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:20:03.533024    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:20:15.440238 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:20:15.440205    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:20:23.533526 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:20:23.533488    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:22:45.356287 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:22:45.356250    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:22:55.966196 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:22:55.966133    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:23:18.872256 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:23:18.872215    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:23:36.650136 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:23:36.650059    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:23:46.167665 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:23:46.167624    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:23:58.374277 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:23:58.374240    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:24:10.156173 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:24:10.156138    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:24:30.560394 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:24:30.560362    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:24:40.743986 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:24:40.743954    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:24:52.248926 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:24:52.248883    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:25:02.345677 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:02.345576    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:25:22.743506 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:22.743468    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:25:29.646763 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.646729    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"]
Jul 30 13:25:29.647207 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.647059    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68f6d126-b213-4649-95c8-65727d0695cd" containerName="cleanup"
Jul 30 13:25:29.647207 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.647069    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="68f6d126-b213-4649-95c8-65727d0695cd" containerName="cleanup"
Jul 30 13:25:29.647207 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.647132    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="68f6d126-b213-4649-95c8-65727d0695cd" containerName="cleanup"
Jul 30 13:25:29.650087 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.650070    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.652972 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.652952    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6m8gs\""
Jul 30 13:25:29.653293 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.653276    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 13:25:29.654430 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.654402    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-378c6e33-kserve-self-signed-certs\""
Jul 30 13:25:29.654547 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.654508    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 13:25:29.670282 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.670253    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"]
Jul 30 13:25:29.810464 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.810435    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-home\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.810667 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.810474    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-dshm\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.810667 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.810534    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-model-cache\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.810667 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.810557    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-tls-certs\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.810667 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.810611    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh8hh\" (UniqueName: \"kubernetes.io/projected/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-kube-api-access-zh8hh\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.810667 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.810659    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-tmp-dir\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.911967 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.911939    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh8hh\" (UniqueName: \"kubernetes.io/projected/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-kube-api-access-zh8hh\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.912143 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.911985    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-tmp-dir\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.912143 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.912011    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-home\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.912143 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.912028    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-dshm\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.912143 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.912072    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-model-cache\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.912143 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.912086    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-tls-certs\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.912393 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.912366    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-tmp-dir\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.912393 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.912379    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-home\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.912519 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.912497    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-model-cache\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.914232 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.914205    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-dshm\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.914530 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.914512    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-tls-certs\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.920435 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.920413    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh8hh\" (UniqueName: \"kubernetes.io/projected/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-kube-api-access-zh8hh\") pod \"e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") " pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:29.959475 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:29.959454    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:30.093022 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:30.092989    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"]
Jul 30 13:25:30.094588 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:25:30.094566    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb33bb4f6_13fa_435c_b4f9_a7fc30e6a378.slice/crio-692b036771e18520cad92ee65f083042632c3f17df3a36c51b2c274c266de5a4 WatchSource:0}: Error finding container 692b036771e18520cad92ee65f083042632c3f17df3a36c51b2c274c266de5a4: Status 404 returned error can't find the container with id 692b036771e18520cad92ee65f083042632c3f17df3a36c51b2c274c266de5a4
Jul 30 13:25:30.096489 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:30.096472    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 13:25:30.842234 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:30.842190    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp" event={"ID":"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378","Type":"ContainerStarted","Data":"692b036771e18520cad92ee65f083042632c3f17df3a36c51b2c274c266de5a4"}
Jul 30 13:25:31.848254 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:31.848216    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp" event={"ID":"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378","Type":"ContainerStarted","Data":"f5d100b1850ac7e4bd64a1eb4e53990b564ee6b8ea95fe12b767800948942e6e"}
Jul 30 13:25:31.866032 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:31.865988    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp" podStartSLOduration=1.713107061 podStartE2EDuration="2.865974401s" totalImagesPullingTime="1.15286734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:25:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:25:31.865554299 +0000 UTC m=+1555.138018697" watchObservedRunningTime="2026-07-30 13:25:31.865974401 +0000 UTC m=+1555.138438798"
Jul 30 13:25:32.141894 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:32.141813    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:25:39.960139 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:39.960104    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:39.960518 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:39.960163    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:39.972378 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:39.972356    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:40.898001 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:40.897970    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:25:54.639369 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:25:54.639338    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:26:10.725255 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:10.725223    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"]
Jul 30 13:26:13.441664 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:13.441623    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:26:25.744243 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:25.744184    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp" podUID="b33bb4f6-13fa-435c-b4f9-a7fc30e6a378" containerName="main" containerID="cri-o://f5d100b1850ac7e4bd64a1eb4e53990b564ee6b8ea95fe12b767800948942e6e" gracePeriod=45
Jul 30 13:26:25.992001 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:25.991979    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:26:26.084587 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.084501    2557 generic.go:356] "Generic (PLEG): container finished" podID="b33bb4f6-13fa-435c-b4f9-a7fc30e6a378" containerID="f5d100b1850ac7e4bd64a1eb4e53990b564ee6b8ea95fe12b767800948942e6e" exitCode=0
Jul 30 13:26:26.084587 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.084546    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"
Jul 30 13:26:26.084587 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.084551    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp" event={"ID":"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378","Type":"ContainerDied","Data":"f5d100b1850ac7e4bd64a1eb4e53990b564ee6b8ea95fe12b767800948942e6e"}
Jul 30 13:26:26.084587 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.084577    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp" event={"ID":"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378","Type":"ContainerDied","Data":"692b036771e18520cad92ee65f083042632c3f17df3a36c51b2c274c266de5a4"}
Jul 30 13:26:26.084901 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.084619    2557 scope.go:117] "RemoveContainer" containerID="f5d100b1850ac7e4bd64a1eb4e53990b564ee6b8ea95fe12b767800948942e6e"
Jul 30 13:26:26.093035 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.093010    2557 scope.go:117] "RemoveContainer" containerID="f5d100b1850ac7e4bd64a1eb4e53990b564ee6b8ea95fe12b767800948942e6e"
Jul 30 13:26:26.093276 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:26:26.093258    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5d100b1850ac7e4bd64a1eb4e53990b564ee6b8ea95fe12b767800948942e6e\": container with ID starting with f5d100b1850ac7e4bd64a1eb4e53990b564ee6b8ea95fe12b767800948942e6e not found: ID does not exist" containerID="f5d100b1850ac7e4bd64a1eb4e53990b564ee6b8ea95fe12b767800948942e6e"
Jul 30 13:26:26.093317 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.093285    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5d100b1850ac7e4bd64a1eb4e53990b564ee6b8ea95fe12b767800948942e6e"} err="failed to get container status \"f5d100b1850ac7e4bd64a1eb4e53990b564ee6b8ea95fe12b767800948942e6e\": rpc error: code = NotFound desc = could not find container \"f5d100b1850ac7e4bd64a1eb4e53990b564ee6b8ea95fe12b767800948942e6e\": container with ID starting with f5d100b1850ac7e4bd64a1eb4e53990b564ee6b8ea95fe12b767800948942e6e not found: ID does not exist"
Jul 30 13:26:26.099653 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.099637    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-dshm\") pod \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") "
Jul 30 13:26:26.099723 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.099662    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-tls-certs\") pod \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") "
Jul 30 13:26:26.099723 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.099681    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-home\") pod \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") "
Jul 30 13:26:26.099831 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.099725    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-model-cache\") pod \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") "
Jul 30 13:26:26.099831 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.099789    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh8hh\" (UniqueName: \"kubernetes.io/projected/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-kube-api-access-zh8hh\") pod \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") "
Jul 30 13:26:26.099934 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.099837    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-tmp-dir\") pod \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\" (UID: \"b33bb4f6-13fa-435c-b4f9-a7fc30e6a378\") "
Jul 30 13:26:26.099985 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.099934    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-home" (OuterVolumeSpecName: "home") pod "b33bb4f6-13fa-435c-b4f9-a7fc30e6a378" (UID: "b33bb4f6-13fa-435c-b4f9-a7fc30e6a378"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:26:26.100029 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.099993    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-model-cache" (OuterVolumeSpecName: "model-cache") pod "b33bb4f6-13fa-435c-b4f9-a7fc30e6a378" (UID: "b33bb4f6-13fa-435c-b4f9-a7fc30e6a378"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:26:26.100153 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.100139    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-home\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:26:26.100153 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.100136    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b33bb4f6-13fa-435c-b4f9-a7fc30e6a378" (UID: "b33bb4f6-13fa-435c-b4f9-a7fc30e6a378"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:26:26.100230 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.100158    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-model-cache\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:26:26.101817 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.101795    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b33bb4f6-13fa-435c-b4f9-a7fc30e6a378" (UID: "b33bb4f6-13fa-435c-b4f9-a7fc30e6a378"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 13:26:26.101907 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.101835    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-dshm" (OuterVolumeSpecName: "dshm") pod "b33bb4f6-13fa-435c-b4f9-a7fc30e6a378" (UID: "b33bb4f6-13fa-435c-b4f9-a7fc30e6a378"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:26:26.101907 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.101873    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-kube-api-access-zh8hh" (OuterVolumeSpecName: "kube-api-access-zh8hh") pod "b33bb4f6-13fa-435c-b4f9-a7fc30e6a378" (UID: "b33bb4f6-13fa-435c-b4f9-a7fc30e6a378"). InnerVolumeSpecName "kube-api-access-zh8hh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:26:26.200547 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.200513    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh8hh\" (UniqueName: \"kubernetes.io/projected/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-kube-api-access-zh8hh\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:26:26.200547 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.200541    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-tmp-dir\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:26:26.200741 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.200556    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-dshm\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:26:26.200741 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.200568    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378-tls-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:26:26.406208 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.406175    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"]
Jul 30 13:26:26.410228 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:26.410200    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-378c6e33-kserve-84f9f9889f-v2pnp"]
Jul 30 13:26:27.183209 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:27.183173    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b33bb4f6-13fa-435c-b4f9-a7fc30e6a378" path="/var/lib/kubelet/pods/b33bb4f6-13fa-435c-b4f9-a7fc30e6a378/volumes"
Jul 30 13:26:27.632137 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:27.632065    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:26:34.548809 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:34.548728    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:26:45.749785 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:26:45.749753    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:27:00.335277 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:27:00.335240    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:27:13.832995 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:27:13.832956    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:27:27.729189 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:27:27.729156    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:27:43.051738 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:27:43.051702    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:27:52.643791 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:27:52.643756    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:28:05.949845 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:28:05.949764    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:28:16.049583 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:28:16.049541    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:28:25.955774 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:28:25.955738    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:28:38.933816 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:28:38.933778    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:28:50.634976 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:28:50.634940    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:28:58.240738 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:28:58.240701    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:29:21.062133 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:29:21.062094    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:29:35.272067 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:29:35.271977    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:29:49.084834 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:29:49.084798    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:30:00.143538 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:00.143500    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756970-czp7p"]
Jul 30 13:30:00.143962 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:00.143883    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b33bb4f6-13fa-435c-b4f9-a7fc30e6a378" containerName="main"
Jul 30 13:30:00.143962 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:00.143897    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b33bb4f6-13fa-435c-b4f9-a7fc30e6a378" containerName="main"
Jul 30 13:30:00.143962 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:00.143950    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b33bb4f6-13fa-435c-b4f9-a7fc30e6a378" containerName="main"
Jul 30 13:30:00.146895 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:00.146879    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756970-czp7p"
Jul 30 13:30:00.149930 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:00.149906    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxdpz\""
Jul 30 13:30:00.156133 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:00.156112    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756970-czp7p"
Jul 30 13:30:00.162047 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:00.162020    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756970-czp7p"]
Jul 30 13:30:00.494046 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:00.494019    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756970-czp7p"]
Jul 30 13:30:00.496496 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:30:00.496469    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe7661ce_26e5_43a3_a727_9e31e654190a.slice/crio-e75a3d0c1b30369545195c1dcb61ff09ce1217d4338170067b63a8cabade1e57 WatchSource:0}: Error finding container e75a3d0c1b30369545195c1dcb61ff09ce1217d4338170067b63a8cabade1e57: Status 404 returned error can't find the container with id e75a3d0c1b30369545195c1dcb61ff09ce1217d4338170067b63a8cabade1e57
Jul 30 13:30:01.023367 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:01.023280    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756970-czp7p" event={"ID":"be7661ce-26e5-43a3-a727-9e31e654190a","Type":"ContainerStarted","Data":"07436b310edf610291dd05f697e5b1984bbdac1e9ed686b2e107fec68daf576a"}
Jul 30 13:30:01.023367 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:01.023329    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756970-czp7p" event={"ID":"be7661ce-26e5-43a3-a727-9e31e654190a","Type":"ContainerStarted","Data":"e75a3d0c1b30369545195c1dcb61ff09ce1217d4338170067b63a8cabade1e57"}
Jul 30 13:30:02.030442 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:02.030409    2557 generic.go:356] "Generic (PLEG): container finished" podID="be7661ce-26e5-43a3-a727-9e31e654190a" containerID="07436b310edf610291dd05f697e5b1984bbdac1e9ed686b2e107fec68daf576a" exitCode=0
Jul 30 13:30:02.030824 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:02.030473    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756970-czp7p" event={"ID":"be7661ce-26e5-43a3-a727-9e31e654190a","Type":"ContainerDied","Data":"07436b310edf610291dd05f697e5b1984bbdac1e9ed686b2e107fec68daf576a"}
Jul 30 13:30:03.171985 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:03.171954    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756970-czp7p"
Jul 30 13:30:04.040413 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:04.040389    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756970-czp7p"
Jul 30 13:30:04.040592 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:04.040386    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756970-czp7p" event={"ID":"be7661ce-26e5-43a3-a727-9e31e654190a","Type":"ContainerDied","Data":"e75a3d0c1b30369545195c1dcb61ff09ce1217d4338170067b63a8cabade1e57"}
Jul 30 13:30:04.040592 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:04.040519    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e75a3d0c1b30369545195c1dcb61ff09ce1217d4338170067b63a8cabade1e57"
Jul 30 13:30:04.046050 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:04.046025    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:30:13.430715 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:13.430675    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:30:27.643172 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:27.643139    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:30:41.362761 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:41.362728    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:30:51.242990 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:30:51.242947    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:31:01.936802 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:01.936728    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:31:03.245390 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.245352    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"]
Jul 30 13:31:03.245797 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.245733    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be7661ce-26e5-43a3-a727-9e31e654190a" containerName="cleanup"
Jul 30 13:31:03.245797 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.245745    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="be7661ce-26e5-43a3-a727-9e31e654190a" containerName="cleanup"
Jul 30 13:31:03.245913 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.245799    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="be7661ce-26e5-43a3-a727-9e31e654190a" containerName="cleanup"
Jul 30 13:31:03.248818 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.248803    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.252524 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.252498    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a593ec13-openshift-default-dockercfg-plvqh\""
Jul 30 13:31:03.252682 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.252552    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 13:31:03.284341 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.284312    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"]
Jul 30 13:31:03.399610 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.399574    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-istio-envoy\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.399780 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.399671    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/821b6b42-d3b1-4114-9547-d77569d8badb-istio-podinfo\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.399780 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.399722    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-istio-data\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.399780 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.399740    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/821b6b42-d3b1-4114-9547-d77569d8badb-istiod-ca-cert\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.399780 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.399756    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2q9b\" (UniqueName: \"kubernetes.io/projected/821b6b42-d3b1-4114-9547-d77569d8badb-kube-api-access-g2q9b\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.399987 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.399781    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-credential-socket\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.399987 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.399855    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-workload-socket\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.399987 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.399948    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/821b6b42-d3b1-4114-9547-d77569d8badb-istio-token\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.400110 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.400000    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-workload-certs\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.501081 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.500988    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-workload-socket\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.501081 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.501047    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/821b6b42-d3b1-4114-9547-d77569d8badb-istio-token\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.501081 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.501075    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-workload-certs\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.501368 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.501096    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-istio-envoy\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.501368 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.501119    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/821b6b42-d3b1-4114-9547-d77569d8badb-istio-podinfo\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.501368 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.501279    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-istio-data\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.501368 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.501315    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/821b6b42-d3b1-4114-9547-d77569d8badb-istiod-ca-cert\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.501368 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.501342    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2q9b\" (UniqueName: \"kubernetes.io/projected/821b6b42-d3b1-4114-9547-d77569d8badb-kube-api-access-g2q9b\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.501692 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.501380    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-credential-socket\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.501692 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.501475    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-workload-socket\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.501692 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.501508    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-workload-certs\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.501851 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.501749    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-credential-socket\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.501851 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.501780    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-istio-data\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.502009 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.501990    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/821b6b42-d3b1-4114-9547-d77569d8badb-istiod-ca-cert\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.503450 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.503430    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-istio-envoy\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.503692 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.503676    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/821b6b42-d3b1-4114-9547-d77569d8badb-istio-podinfo\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.512893 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.512867    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2q9b\" (UniqueName: \"kubernetes.io/projected/821b6b42-d3b1-4114-9547-d77569d8badb-kube-api-access-g2q9b\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.513017 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.512985    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/821b6b42-d3b1-4114-9547-d77569d8badb-istio-token\") pod \"e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") " pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.558872 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.558839    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:03.698114 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.698086    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"]
Jul 30 13:31:03.699896 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:31:03.699870    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod821b6b42_d3b1_4114_9547_d77569d8badb.slice/crio-e73226e549da4e59f9c54ed44a59bffeba792f9cad794564ecde8546dbaecd5e WatchSource:0}: Error finding container e73226e549da4e59f9c54ed44a59bffeba792f9cad794564ecde8546dbaecd5e: Status 404 returned error can't find the container with id e73226e549da4e59f9c54ed44a59bffeba792f9cad794564ecde8546dbaecd5e
Jul 30 13:31:03.701768 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:03.701749    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 13:31:04.298292 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:04.298255    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc" event={"ID":"821b6b42-d3b1-4114-9547-d77569d8badb","Type":"ContainerStarted","Data":"e73226e549da4e59f9c54ed44a59bffeba792f9cad794564ecde8546dbaecd5e"}
Jul 30 13:31:07.262203 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:07.262166    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:31:07.262462 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:07.262238    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:31:07.262462 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:07.262266    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:31:08.317203 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:08.317163    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc" event={"ID":"821b6b42-d3b1-4114-9547-d77569d8badb","Type":"ContainerStarted","Data":"0fc627809aec5994ccdc0956434ddadc8ba21bf7c7235fe1f88217a6a8acb264"}
Jul 30 13:31:08.344092 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:08.344044    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc" podStartSLOduration=1.784012267 podStartE2EDuration="5.344027862s" totalImagesPullingTime="3.560015595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:31:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:31:08.342177241 +0000 UTC m=+1891.614641658" watchObservedRunningTime="2026-07-30 13:31:08.344027862 +0000 UTC m=+1891.616492259"
Jul 30 13:31:08.559337 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:08.559302    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:08.564045 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:08.564018    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:09.322211 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:09.322180    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:09.323135 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:09.323117    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:16.451978 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:16.451941    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:31:24.523719 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:24.523674    2557 patch_prober.go:28] interesting pod/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": read tcp 10.134.0.2:37654->10.134.0.29:15021: read: connection reset by peer" start-of-body=
Jul 30 13:31:24.524133 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:24.523757    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc" podUID="821b6b42-d3b1-4114-9547-d77569d8badb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": read tcp 10.134.0.2:37654->10.134.0.29:15021: read: connection reset by peer"
Jul 30 13:31:25.405193 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:25.405159    2557 generic.go:356] "Generic (PLEG): container finished" podID="821b6b42-d3b1-4114-9547-d77569d8badb" containerID="0fc627809aec5994ccdc0956434ddadc8ba21bf7c7235fe1f88217a6a8acb264" exitCode=0
Jul 30 13:31:25.405382 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:25.405234    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc" event={"ID":"821b6b42-d3b1-4114-9547-d77569d8badb","Type":"ContainerDied","Data":"0fc627809aec5994ccdc0956434ddadc8ba21bf7c7235fe1f88217a6a8acb264"}
Jul 30 13:31:25.405700 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:25.405681    2557 scope.go:117] "RemoveContainer" containerID="0fc627809aec5994ccdc0956434ddadc8ba21bf7c7235fe1f88217a6a8acb264"
Jul 30 13:31:25.406735 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:25.406706    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:31:25.406818 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:25.406784    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:31:25.406858 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:25.406813    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:31:26.412029 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:26.411993    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc" event={"ID":"821b6b42-d3b1-4114-9547-d77569d8badb","Type":"ContainerStarted","Data":"88d1727e14a486457a6b29c72f3605b9aa51911346965761565790280da39bd0"}
Jul 30 13:31:26.559921 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:26.559884    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:26.560215 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:26.560193    2557 patch_prober.go:28] interesting pod/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Jul 30 13:31:26.560301 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:26.560247    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc" podUID="821b6b42-d3b1-4114-9547-d77569d8badb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Jul 30 13:31:27.559982 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:27.559946    2557 patch_prober.go:28] interesting pod/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Jul 30 13:31:27.560366 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:27.560016    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc" podUID="821b6b42-d3b1-4114-9547-d77569d8badb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Jul 30 13:31:28.559472 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:28.559434    2557 patch_prober.go:28] interesting pod/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Jul 30 13:31:28.559698 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:28.559496    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc" podUID="821b6b42-d3b1-4114-9547-d77569d8badb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Jul 30 13:31:29.559681 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:29.559646    2557 patch_prober.go:28] interesting pod/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Jul 30 13:31:29.560059 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:29.559712    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc" podUID="821b6b42-d3b1-4114-9547-d77569d8badb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Jul 30 13:31:30.562887 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:30.562859    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:30.563255 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:30.563177    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:30.563871 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:30.563855    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:35.491785 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:35.491751    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26"]
Jul 30 13:31:35.498405 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:35.498384    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26"
Jul 30 13:31:35.502475 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:35.502454    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxdpz\""
Jul 30 13:31:35.502890 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:35.502875    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 13:31:35.508670 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:35.508649    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26"]
Jul 30 13:31:35.583546 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:35.583509    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea98efb-9e4f-49f1-893e-6cd97257ab06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a593ec13-nmv26\" (UID: \"6ea98efb-9e4f-49f1-893e-6cd97257ab06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26"
Jul 30 13:31:35.684210 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:35.684175    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea98efb-9e4f-49f1-893e-6cd97257ab06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a593ec13-nmv26\" (UID: \"6ea98efb-9e4f-49f1-893e-6cd97257ab06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26"
Jul 30 13:31:35.684852 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:35.684832    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea98efb-9e4f-49f1-893e-6cd97257ab06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a593ec13-nmv26\" (UID: \"6ea98efb-9e4f-49f1-893e-6cd97257ab06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26"
Jul 30 13:31:35.809520 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:35.809436    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26"
Jul 30 13:31:35.962078 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:35.962046    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26"]
Jul 30 13:31:35.971163 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:31:35.971100    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ea98efb_9e4f_49f1_893e_6cd97257ab06.slice/crio-bc30dd3f90db37bec2c563534712754bb01e6de7a04e1e904202a56c1186df23 WatchSource:0}: Error finding container bc30dd3f90db37bec2c563534712754bb01e6de7a04e1e904202a56c1186df23: Status 404 returned error can't find the container with id bc30dd3f90db37bec2c563534712754bb01e6de7a04e1e904202a56c1186df23
Jul 30 13:31:36.459520 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:36.459426    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26" event={"ID":"6ea98efb-9e4f-49f1-893e-6cd97257ab06","Type":"ContainerStarted","Data":"40e95e23d64a102d800c8eb7eb6b4b6825bf385d8faa32afc692d8e6b04759c8"}
Jul 30 13:31:36.459520 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:36.459485    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26" event={"ID":"6ea98efb-9e4f-49f1-893e-6cd97257ab06","Type":"ContainerStarted","Data":"bc30dd3f90db37bec2c563534712754bb01e6de7a04e1e904202a56c1186df23"}
Jul 30 13:31:36.478331 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:36.478286    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26" podStartSLOduration=1.478271259 podStartE2EDuration="1.478271259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:31:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:31:36.476899426 +0000 UTC m=+1919.749363851" watchObservedRunningTime="2026-07-30 13:31:36.478271259 +0000 UTC m=+1919.750735656"
Jul 30 13:31:37.465219 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:37.465185    2557 generic.go:356] "Generic (PLEG): container finished" podID="6ea98efb-9e4f-49f1-893e-6cd97257ab06" containerID="40e95e23d64a102d800c8eb7eb6b4b6825bf385d8faa32afc692d8e6b04759c8" exitCode=0
Jul 30 13:31:37.465649 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:37.465275    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26" event={"ID":"6ea98efb-9e4f-49f1-893e-6cd97257ab06","Type":"ContainerDied","Data":"40e95e23d64a102d800c8eb7eb6b4b6825bf385d8faa32afc692d8e6b04759c8"}
Jul 30 13:31:38.600209 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:38.600186    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26"
Jul 30 13:31:38.713570 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:38.713538    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea98efb-9e4f-49f1-893e-6cd97257ab06-maas-api-service-ca\") pod \"6ea98efb-9e4f-49f1-893e-6cd97257ab06\" (UID: \"6ea98efb-9e4f-49f1-893e-6cd97257ab06\") "
Jul 30 13:31:38.713901 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:38.713877    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ea98efb-9e4f-49f1-893e-6cd97257ab06-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6ea98efb-9e4f-49f1-893e-6cd97257ab06" (UID: "6ea98efb-9e4f-49f1-893e-6cd97257ab06"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:31:38.814962 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:38.814881    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea98efb-9e4f-49f1-893e-6cd97257ab06-maas-api-service-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:31:39.476478 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:39.476384    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26" event={"ID":"6ea98efb-9e4f-49f1-893e-6cd97257ab06","Type":"ContainerDied","Data":"bc30dd3f90db37bec2c563534712754bb01e6de7a04e1e904202a56c1186df23"}
Jul 30 13:31:39.476478 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:39.476437    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc30dd3f90db37bec2c563534712754bb01e6de7a04e1e904202a56c1186df23"
Jul 30 13:31:39.476478 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:39.476401    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26"
Jul 30 13:31:41.877379 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:41.877345    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26"]
Jul 30 13:31:41.882523 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:41.882497    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a593ec13-nmv26"]
Jul 30 13:31:42.045049 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:42.045017    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"]
Jul 30 13:31:42.045245 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:42.045219    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc" podUID="821b6b42-d3b1-4114-9547-d77569d8badb" containerName="istio-proxy" containerID="cri-o://88d1727e14a486457a6b29c72f3605b9aa51911346965761565790280da39bd0" gracePeriod=30
Jul 30 13:31:43.131102 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.131067    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"]
Jul 30 13:31:43.131463 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.131393    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ea98efb-9e4f-49f1-893e-6cd97257ab06" containerName="revoke-keys"
Jul 30 13:31:43.131463 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.131403    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ea98efb-9e4f-49f1-893e-6cd97257ab06" containerName="revoke-keys"
Jul 30 13:31:43.131532 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.131477    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ea98efb-9e4f-49f1-893e-6cd97257ab06" containerName="revoke-keys"
Jul 30 13:31:43.134387 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.134371    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.140822 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.140796    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-c5e6a00f-gw-openshift-default-dockercfg-twnww\""
Jul 30 13:31:43.160576 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.160548    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"]
Jul 30 13:31:43.184494 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.184464    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ea98efb-9e4f-49f1-893e-6cd97257ab06" path="/var/lib/kubelet/pods/6ea98efb-9e4f-49f1-893e-6cd97257ab06/volumes"
Jul 30 13:31:43.255451 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.255410    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9569c782-fd52-488e-81a5-27560c649ef1-istio-token\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.255639 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.255470    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-credential-socket\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.255639 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.255497    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9569c782-fd52-488e-81a5-27560c649ef1-istio-podinfo\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.255639 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.255532    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-istio-envoy\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.255639 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.255547    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs8mb\" (UniqueName: \"kubernetes.io/projected/9569c782-fd52-488e-81a5-27560c649ef1-kube-api-access-zs8mb\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.255639 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.255571    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-workload-certs\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.255639 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.255617    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-workload-socket\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.255838 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.255691    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-istio-data\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.255838 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.255725    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9569c782-fd52-488e-81a5-27560c649ef1-istiod-ca-cert\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.357095 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.357067    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-credential-socket\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.357095 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.357101    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9569c782-fd52-488e-81a5-27560c649ef1-istio-podinfo\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.357303 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.357138    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-istio-envoy\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.357303 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.357294    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs8mb\" (UniqueName: \"kubernetes.io/projected/9569c782-fd52-488e-81a5-27560c649ef1-kube-api-access-zs8mb\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.357390 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.357335    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-workload-certs\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.357390 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.357361    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-workload-socket\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.357488 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.357410    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-istio-data\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.357488 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.357428    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9569c782-fd52-488e-81a5-27560c649ef1-istiod-ca-cert\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.357585 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.357503    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-credential-socket\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.357585 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.357506    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9569c782-fd52-488e-81a5-27560c649ef1-istio-token\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.357904 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.357702    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-workload-certs\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.357904 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.357857    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-istio-data\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.357981 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.357965    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-workload-socket\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.358290 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.358268    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9569c782-fd52-488e-81a5-27560c649ef1-istiod-ca-cert\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.359627 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.359579    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-istio-envoy\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.359782 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.359766    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9569c782-fd52-488e-81a5-27560c649ef1-istio-podinfo\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.374664 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.370352    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9569c782-fd52-488e-81a5-27560c649ef1-istio-token\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.374664 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.370714    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs8mb\" (UniqueName: \"kubernetes.io/projected/9569c782-fd52-488e-81a5-27560c649ef1-kube-api-access-zs8mb\") pod \"e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") " pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.444259 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.444225    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:43.591757 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.591724    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"]
Jul 30 13:31:43.595056 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:31:43.595030    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9569c782_fd52_488e_81a5_27560c649ef1.slice/crio-37818bc5b23a4cf9671aebcc49b3a44faf2617d22972598a88d16e596c8d58c5 WatchSource:0}: Error finding container 37818bc5b23a4cf9671aebcc49b3a44faf2617d22972598a88d16e596c8d58c5: Status 404 returned error can't find the container with id 37818bc5b23a4cf9671aebcc49b3a44faf2617d22972598a88d16e596c8d58c5
Jul 30 13:31:43.597272 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.597242    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:31:43.597341 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.597302    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:31:43.597341 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:43.597329    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:31:44.500143 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:44.500110    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9" event={"ID":"9569c782-fd52-488e-81a5-27560c649ef1","Type":"ContainerStarted","Data":"c9c7a58ae9ce65a8142a0791b5158b28d7e0a2bdce7da5e01446cf7b2e6affac"}
Jul 30 13:31:44.500543 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:44.500162    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9" event={"ID":"9569c782-fd52-488e-81a5-27560c649ef1","Type":"ContainerStarted","Data":"37818bc5b23a4cf9671aebcc49b3a44faf2617d22972598a88d16e596c8d58c5"}
Jul 30 13:31:44.541908 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:44.541863    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9" podStartSLOduration=1.541846737 podStartE2EDuration="1.541846737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:31:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:31:44.539215107 +0000 UTC m=+1927.811679504" watchObservedRunningTime="2026-07-30 13:31:44.541846737 +0000 UTC m=+1927.814311135"
Jul 30 13:31:45.444983 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:45.444942    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:45.449734 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:45.449709    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:45.504340 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:45.504312    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:45.505024 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:45.505006    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:31:45.565773 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:45.565738    2557 patch_prober.go:28] interesting pod/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 13:31:45.565927 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:45.565800    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc" podUID="821b6b42-d3b1-4114-9547-d77569d8badb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 13:31:45.840973 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:45.840884    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:31:47.306565 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.306510    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:47.396532 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.396503    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/821b6b42-d3b1-4114-9547-d77569d8badb-istio-podinfo\") pod \"821b6b42-d3b1-4114-9547-d77569d8badb\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") "
Jul 30 13:31:47.396695 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.396549    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-workload-certs\") pod \"821b6b42-d3b1-4114-9547-d77569d8badb\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") "
Jul 30 13:31:47.396695 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.396570    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2q9b\" (UniqueName: \"kubernetes.io/projected/821b6b42-d3b1-4114-9547-d77569d8badb-kube-api-access-g2q9b\") pod \"821b6b42-d3b1-4114-9547-d77569d8badb\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") "
Jul 30 13:31:47.396784 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.396689    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/821b6b42-d3b1-4114-9547-d77569d8badb-istio-token\") pod \"821b6b42-d3b1-4114-9547-d77569d8badb\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") "
Jul 30 13:31:47.396784 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.396731    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/821b6b42-d3b1-4114-9547-d77569d8badb-istiod-ca-cert\") pod \"821b6b42-d3b1-4114-9547-d77569d8badb\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") "
Jul 30 13:31:47.396784 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.396764    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-istio-envoy\") pod \"821b6b42-d3b1-4114-9547-d77569d8badb\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") "
Jul 30 13:31:47.396918 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.396791    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-workload-socket\") pod \"821b6b42-d3b1-4114-9547-d77569d8badb\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") "
Jul 30 13:31:47.396918 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.396799    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "821b6b42-d3b1-4114-9547-d77569d8badb" (UID: "821b6b42-d3b1-4114-9547-d77569d8badb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:31:47.396918 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.396835    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-istio-data\") pod \"821b6b42-d3b1-4114-9547-d77569d8badb\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") "
Jul 30 13:31:47.396918 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.396859    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-credential-socket\") pod \"821b6b42-d3b1-4114-9547-d77569d8badb\" (UID: \"821b6b42-d3b1-4114-9547-d77569d8badb\") "
Jul 30 13:31:47.397236 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.397193    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/821b6b42-d3b1-4114-9547-d77569d8badb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "821b6b42-d3b1-4114-9547-d77569d8badb" (UID: "821b6b42-d3b1-4114-9547-d77569d8badb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:31:47.397236 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.397222    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-workload-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:31:47.397442 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.397422    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "821b6b42-d3b1-4114-9547-d77569d8badb" (UID: "821b6b42-d3b1-4114-9547-d77569d8badb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:31:47.397675 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.397649    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "821b6b42-d3b1-4114-9547-d77569d8badb" (UID: "821b6b42-d3b1-4114-9547-d77569d8badb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:31:47.397942 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.397916    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-istio-data" (OuterVolumeSpecName: "istio-data") pod "821b6b42-d3b1-4114-9547-d77569d8badb" (UID: "821b6b42-d3b1-4114-9547-d77569d8badb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:31:47.399029 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.399008    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/821b6b42-d3b1-4114-9547-d77569d8badb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "821b6b42-d3b1-4114-9547-d77569d8badb" (UID: "821b6b42-d3b1-4114-9547-d77569d8badb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 13:31:47.399196 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.399176    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/821b6b42-d3b1-4114-9547-d77569d8badb-kube-api-access-g2q9b" (OuterVolumeSpecName: "kube-api-access-g2q9b") pod "821b6b42-d3b1-4114-9547-d77569d8badb" (UID: "821b6b42-d3b1-4114-9547-d77569d8badb"). InnerVolumeSpecName "kube-api-access-g2q9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:31:47.399260 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.399212    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/821b6b42-d3b1-4114-9547-d77569d8badb-istio-token" (OuterVolumeSpecName: "istio-token") pod "821b6b42-d3b1-4114-9547-d77569d8badb" (UID: "821b6b42-d3b1-4114-9547-d77569d8badb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:31:47.399260 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.399214    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "821b6b42-d3b1-4114-9547-d77569d8badb" (UID: "821b6b42-d3b1-4114-9547-d77569d8badb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:31:47.498658 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.498625    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/821b6b42-d3b1-4114-9547-d77569d8badb-istio-token\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:31:47.498658 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.498650    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/821b6b42-d3b1-4114-9547-d77569d8badb-istiod-ca-cert\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:31:47.498658 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.498659    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-istio-envoy\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:31:47.498658 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.498668    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-workload-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:31:47.498923 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.498676    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-istio-data\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:31:47.498923 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.498686    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b6b42-d3b1-4114-9547-d77569d8badb-credential-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:31:47.498923 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.498695    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/821b6b42-d3b1-4114-9547-d77569d8badb-istio-podinfo\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:31:47.498923 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.498703    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2q9b\" (UniqueName: \"kubernetes.io/projected/821b6b42-d3b1-4114-9547-d77569d8badb-kube-api-access-g2q9b\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:31:47.513858 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.513823    2557 generic.go:356] "Generic (PLEG): container finished" podID="821b6b42-d3b1-4114-9547-d77569d8badb" containerID="88d1727e14a486457a6b29c72f3605b9aa51911346965761565790280da39bd0" exitCode=0
Jul 30 13:31:47.514004 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.513874    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc" event={"ID":"821b6b42-d3b1-4114-9547-d77569d8badb","Type":"ContainerDied","Data":"88d1727e14a486457a6b29c72f3605b9aa51911346965761565790280da39bd0"}
Jul 30 13:31:47.514004 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.513885    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"
Jul 30 13:31:47.514004 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.513935    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc" event={"ID":"821b6b42-d3b1-4114-9547-d77569d8badb","Type":"ContainerDied","Data":"e73226e549da4e59f9c54ed44a59bffeba792f9cad794564ecde8546dbaecd5e"}
Jul 30 13:31:47.514004 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.513963    2557 scope.go:117] "RemoveContainer" containerID="88d1727e14a486457a6b29c72f3605b9aa51911346965761565790280da39bd0"
Jul 30 13:31:47.522946 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.522929    2557 scope.go:117] "RemoveContainer" containerID="0fc627809aec5994ccdc0956434ddadc8ba21bf7c7235fe1f88217a6a8acb264"
Jul 30 13:31:47.530741 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.530723    2557 scope.go:117] "RemoveContainer" containerID="88d1727e14a486457a6b29c72f3605b9aa51911346965761565790280da39bd0"
Jul 30 13:31:47.531019 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:31:47.530994    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88d1727e14a486457a6b29c72f3605b9aa51911346965761565790280da39bd0\": container with ID starting with 88d1727e14a486457a6b29c72f3605b9aa51911346965761565790280da39bd0 not found: ID does not exist" containerID="88d1727e14a486457a6b29c72f3605b9aa51911346965761565790280da39bd0"
Jul 30 13:31:47.531107 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.531025    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88d1727e14a486457a6b29c72f3605b9aa51911346965761565790280da39bd0"} err="failed to get container status \"88d1727e14a486457a6b29c72f3605b9aa51911346965761565790280da39bd0\": rpc error: code = NotFound desc = could not find container \"88d1727e14a486457a6b29c72f3605b9aa51911346965761565790280da39bd0\": container with ID starting with 88d1727e14a486457a6b29c72f3605b9aa51911346965761565790280da39bd0 not found: ID does not exist"
Jul 30 13:31:47.531107 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.531043    2557 scope.go:117] "RemoveContainer" containerID="0fc627809aec5994ccdc0956434ddadc8ba21bf7c7235fe1f88217a6a8acb264"
Jul 30 13:31:47.531265 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:31:47.531247    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fc627809aec5994ccdc0956434ddadc8ba21bf7c7235fe1f88217a6a8acb264\": container with ID starting with 0fc627809aec5994ccdc0956434ddadc8ba21bf7c7235fe1f88217a6a8acb264 not found: ID does not exist" containerID="0fc627809aec5994ccdc0956434ddadc8ba21bf7c7235fe1f88217a6a8acb264"
Jul 30 13:31:47.531301 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.531270    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc627809aec5994ccdc0956434ddadc8ba21bf7c7235fe1f88217a6a8acb264"} err="failed to get container status \"0fc627809aec5994ccdc0956434ddadc8ba21bf7c7235fe1f88217a6a8acb264\": rpc error: code = NotFound desc = could not find container \"0fc627809aec5994ccdc0956434ddadc8ba21bf7c7235fe1f88217a6a8acb264\": container with ID starting with 0fc627809aec5994ccdc0956434ddadc8ba21bf7c7235fe1f88217a6a8acb264 not found: ID does not exist"
Jul 30 13:31:47.569892 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.569835    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"]
Jul 30 13:31:47.607162 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:47.607134    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a593ec13-openshift-default-54796f78cb-7gjkc"]
Jul 30 13:31:49.183505 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:49.183472    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="821b6b42-d3b1-4114-9547-d77569d8badb" path="/var/lib/kubelet/pods/821b6b42-d3b1-4114-9547-d77569d8badb/volumes"
Jul 30 13:31:56.547479 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:31:56.547444    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:32:00.506043 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:00.506010    2557 patch_prober.go:28] interesting pod/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 30 13:32:00.506491 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:00.506061    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9" podUID="9569c782-fd52-488e-81a5-27560c649ef1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 30 13:32:00.580588 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:00.580552    2557 generic.go:356] "Generic (PLEG): container finished" podID="9569c782-fd52-488e-81a5-27560c649ef1" containerID="c9c7a58ae9ce65a8142a0791b5158b28d7e0a2bdce7da5e01446cf7b2e6affac" exitCode=0
Jul 30 13:32:00.580761 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:00.580629    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9" event={"ID":"9569c782-fd52-488e-81a5-27560c649ef1","Type":"ContainerDied","Data":"c9c7a58ae9ce65a8142a0791b5158b28d7e0a2bdce7da5e01446cf7b2e6affac"}
Jul 30 13:32:00.581048 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:00.581033    2557 scope.go:117] "RemoveContainer" containerID="c9c7a58ae9ce65a8142a0791b5158b28d7e0a2bdce7da5e01446cf7b2e6affac"
Jul 30 13:32:00.582108 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:00.582077    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:32:00.582222 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:00.582148    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:32:00.582222 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:00.582197    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:32:01.587233 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:01.587203    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9" event={"ID":"9569c782-fd52-488e-81a5-27560c649ef1","Type":"ContainerStarted","Data":"8cdfe6ec44bef6bc788e5ccd541e32cb962266926398ba1ed621c7e23d3a3bcc"}
Jul 30 13:32:02.444635 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:02.444590    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:32:02.444831 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:02.444808    2557 patch_prober.go:28] interesting pod/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 30 13:32:02.444882 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:02.444864    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9" podUID="9569c782-fd52-488e-81a5-27560c649ef1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 30 13:32:03.444576 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:03.444537    2557 patch_prober.go:28] interesting pod/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 30 13:32:03.444990 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:03.444628    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9" podUID="9569c782-fd52-488e-81a5-27560c649ef1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 30 13:32:04.445108 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:04.445065    2557 patch_prober.go:28] interesting pod/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 30 13:32:04.445507 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:04.445149    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9" podUID="9569c782-fd52-488e-81a5-27560c649ef1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 30 13:32:05.445018 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:05.444984    2557 patch_prober.go:28] interesting pod/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 30 13:32:05.445189 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:05.445037    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9" podUID="9569c782-fd52-488e-81a5-27560c649ef1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 30 13:32:06.448307 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:06.448270    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:32:06.448712 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:06.448616    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:32:06.449241 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:06.449225    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:32:11.068730 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.068691    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb"]
Jul 30 13:32:11.069235 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.069196    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="821b6b42-d3b1-4114-9547-d77569d8badb" containerName="istio-proxy"
Jul 30 13:32:11.069235 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.069214    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="821b6b42-d3b1-4114-9547-d77569d8badb" containerName="istio-proxy"
Jul 30 13:32:11.069316 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.069240    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="821b6b42-d3b1-4114-9547-d77569d8badb" containerName="istio-proxy"
Jul 30 13:32:11.069316 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.069246    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="821b6b42-d3b1-4114-9547-d77569d8badb" containerName="istio-proxy"
Jul 30 13:32:11.069316 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.069305    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="821b6b42-d3b1-4114-9547-d77569d8badb" containerName="istio-proxy"
Jul 30 13:32:11.072143 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.072126    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb"
Jul 30 13:32:11.075386 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.075363    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 13:32:11.075502 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.075419    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxdpz\""
Jul 30 13:32:11.081584 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.081558    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb"]
Jul 30 13:32:11.109318 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.109290    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1571df30-95ba-4ff9-a442-21daf0d6494b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb\" (UID: \"1571df30-95ba-4ff9-a442-21daf0d6494b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb"
Jul 30 13:32:11.210075 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.210043    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1571df30-95ba-4ff9-a442-21daf0d6494b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb\" (UID: \"1571df30-95ba-4ff9-a442-21daf0d6494b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb"
Jul 30 13:32:11.210706 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.210685    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1571df30-95ba-4ff9-a442-21daf0d6494b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb\" (UID: \"1571df30-95ba-4ff9-a442-21daf0d6494b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb"
Jul 30 13:32:11.383240 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.383156    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb"
Jul 30 13:32:11.520715 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.520680    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb"]
Jul 30 13:32:11.526305 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:32:11.526274    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1571df30_95ba_4ff9_a442_21daf0d6494b.slice/crio-c97e4ab661d7ab134f2b9c1fe5edab609a3714973998476fe4a73aff11022011 WatchSource:0}: Error finding container c97e4ab661d7ab134f2b9c1fe5edab609a3714973998476fe4a73aff11022011: Status 404 returned error can't find the container with id c97e4ab661d7ab134f2b9c1fe5edab609a3714973998476fe4a73aff11022011
Jul 30 13:32:11.632015 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:11.631980    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb" event={"ID":"1571df30-95ba-4ff9-a442-21daf0d6494b","Type":"ContainerStarted","Data":"c97e4ab661d7ab134f2b9c1fe5edab609a3714973998476fe4a73aff11022011"}
Jul 30 13:32:12.638376 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:12.638287    2557 generic.go:356] "Generic (PLEG): container finished" podID="1571df30-95ba-4ff9-a442-21daf0d6494b" containerID="7b3c712e2bb95166aaa3877459b3900749fabf86e1371c616d5d05a18840ad1b" exitCode=0
Jul 30 13:32:12.638732 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:12.638365    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb" event={"ID":"1571df30-95ba-4ff9-a442-21daf0d6494b","Type":"ContainerDied","Data":"7b3c712e2bb95166aaa3877459b3900749fabf86e1371c616d5d05a18840ad1b"}
Jul 30 13:32:13.771929 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:13.771903    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb"
Jul 30 13:32:13.834845 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:13.834807    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1571df30-95ba-4ff9-a442-21daf0d6494b-maas-api-service-ca\") pod \"1571df30-95ba-4ff9-a442-21daf0d6494b\" (UID: \"1571df30-95ba-4ff9-a442-21daf0d6494b\") "
Jul 30 13:32:13.835150 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:13.835129    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1571df30-95ba-4ff9-a442-21daf0d6494b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1571df30-95ba-4ff9-a442-21daf0d6494b" (UID: "1571df30-95ba-4ff9-a442-21daf0d6494b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:32:13.935703 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:13.935672    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1571df30-95ba-4ff9-a442-21daf0d6494b-maas-api-service-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:14.648611 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:14.648569    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb"
Jul 30 13:32:14.648786 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:14.648626    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb" event={"ID":"1571df30-95ba-4ff9-a442-21daf0d6494b","Type":"ContainerDied","Data":"c97e4ab661d7ab134f2b9c1fe5edab609a3714973998476fe4a73aff11022011"}
Jul 30 13:32:14.648786 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:14.648670    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c97e4ab661d7ab134f2b9c1fe5edab609a3714973998476fe4a73aff11022011"
Jul 30 13:32:17.276319 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:17.276288    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb"]
Jul 30 13:32:17.281425 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:17.281402    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c5e6a00f-swzhb"]
Jul 30 13:32:17.415372 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:17.415337    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"]
Jul 30 13:32:17.415565 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:17.415549    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9" podUID="9569c782-fd52-488e-81a5-27560c649ef1" containerName="istio-proxy" containerID="cri-o://8cdfe6ec44bef6bc788e5ccd541e32cb962266926398ba1ed621c7e23d3a3bcc" gracePeriod=30
Jul 30 13:32:18.284498 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.284463    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"]
Jul 30 13:32:18.284901 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.284818    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1571df30-95ba-4ff9-a442-21daf0d6494b" containerName="revoke-keys"
Jul 30 13:32:18.284901 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.284829    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1571df30-95ba-4ff9-a442-21daf0d6494b" containerName="revoke-keys"
Jul 30 13:32:18.284901 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.284890    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1571df30-95ba-4ff9-a442-21daf0d6494b" containerName="revoke-keys"
Jul 30 13:32:18.284901 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.284902    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="821b6b42-d3b1-4114-9547-d77569d8badb" containerName="istio-proxy"
Jul 30 13:32:18.288108 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.288089    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.293466 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.293442    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c625014c-openshift-default-dockercfg-p9rsz\""
Jul 30 13:32:18.317412 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.317381    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"]
Jul 30 13:32:18.374129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.374092    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-workload-socket\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.374129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.374131    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-token\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.374385 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.374163    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-envoy\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.374385 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.374282    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-credential-socket\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.374385 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.374319    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-workload-certs\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.374385 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.374349    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-data\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.374385 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.374374    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istiod-ca-cert\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.374585 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.374405    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-podinfo\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.374585 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.374453    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5gqg\" (UniqueName: \"kubernetes.io/projected/5a42b68f-9e61-4c25-87fa-b052a1435bc8-kube-api-access-n5gqg\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.474878 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.474843    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-envoy\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.475047 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.474909    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-credential-socket\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.475047 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.475018    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-workload-certs\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.475146 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.475055    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-data\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.475146 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.475071    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istiod-ca-cert\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.475251 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.475155    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-podinfo\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.475251 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.475203    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5gqg\" (UniqueName: \"kubernetes.io/projected/5a42b68f-9e61-4c25-87fa-b052a1435bc8-kube-api-access-n5gqg\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.475358 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.475275    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-workload-socket\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.475358 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.475293    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-credential-socket\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.475358 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.475311    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-token\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.475511 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.475394    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-data\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.475511 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.475467    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-workload-certs\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.475705 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.475687    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-workload-socket\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.475788 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.475770    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istiod-ca-cert\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.477204 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.477184    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-envoy\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.477731 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.477713    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-podinfo\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.492011 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.491986    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-token\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.496180 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.496161    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5gqg\" (UniqueName: \"kubernetes.io/projected/5a42b68f-9e61-4c25-87fa-b052a1435bc8-kube-api-access-n5gqg\") pod \"e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") " pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.599056 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.598976    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:18.743669 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.743641    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"]
Jul 30 13:32:18.744416 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:32:18.744393    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a42b68f_9e61_4c25_87fa_b052a1435bc8.slice/crio-512b89e9e162b340ddd600e8dc7c34312e9d0c673d1f1d09db27dd355bc48202 WatchSource:0}: Error finding container 512b89e9e162b340ddd600e8dc7c34312e9d0c673d1f1d09db27dd355bc48202: Status 404 returned error can't find the container with id 512b89e9e162b340ddd600e8dc7c34312e9d0c673d1f1d09db27dd355bc48202
Jul 30 13:32:18.746508 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.746477    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:32:18.746630 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.746550    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:32:18.746630 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:18.746591    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:32:19.183700 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:19.183668    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1571df30-95ba-4ff9-a442-21daf0d6494b" path="/var/lib/kubelet/pods/1571df30-95ba-4ff9-a442-21daf0d6494b/volumes"
Jul 30 13:32:19.676309 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:19.676278    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw" event={"ID":"5a42b68f-9e61-4c25-87fa-b052a1435bc8","Type":"ContainerStarted","Data":"2b43c81cb9027d54537f8ad78dcbae5a98dd80b0831743bf64560c4ace6d7f61"}
Jul 30 13:32:19.676750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:19.676329    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw" event={"ID":"5a42b68f-9e61-4c25-87fa-b052a1435bc8","Type":"ContainerStarted","Data":"512b89e9e162b340ddd600e8dc7c34312e9d0c673d1f1d09db27dd355bc48202"}
Jul 30 13:32:19.701584 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:19.701531    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw" podStartSLOduration=1.701516445 podStartE2EDuration="1.701516445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:32:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:32:19.699135722 +0000 UTC m=+1962.971600139" watchObservedRunningTime="2026-07-30 13:32:19.701516445 +0000 UTC m=+1962.973980842"
Jul 30 13:32:20.599671 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:20.599634    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:20.604673 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:20.604645    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:20.681368 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:20.681342    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:20.682380 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:20.682358    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:21.270325 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:21.270291    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:32:21.451403 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:21.451372    2557 patch_prober.go:28] interesting pod/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 13:32:21.451555 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:21.451422    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9" podUID="9569c782-fd52-488e-81a5-27560c649ef1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 13:32:22.674145 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.674084    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:32:22.693531 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.693500    2557 generic.go:356] "Generic (PLEG): container finished" podID="9569c782-fd52-488e-81a5-27560c649ef1" containerID="8cdfe6ec44bef6bc788e5ccd541e32cb962266926398ba1ed621c7e23d3a3bcc" exitCode=0
Jul 30 13:32:22.693709 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.693554    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"
Jul 30 13:32:22.693709 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.693552    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9" event={"ID":"9569c782-fd52-488e-81a5-27560c649ef1","Type":"ContainerDied","Data":"8cdfe6ec44bef6bc788e5ccd541e32cb962266926398ba1ed621c7e23d3a3bcc"}
Jul 30 13:32:22.693709 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.693672    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9" event={"ID":"9569c782-fd52-488e-81a5-27560c649ef1","Type":"ContainerDied","Data":"37818bc5b23a4cf9671aebcc49b3a44faf2617d22972598a88d16e596c8d58c5"}
Jul 30 13:32:22.693709 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.693703    2557 scope.go:117] "RemoveContainer" containerID="8cdfe6ec44bef6bc788e5ccd541e32cb962266926398ba1ed621c7e23d3a3bcc"
Jul 30 13:32:22.702061 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.702047    2557 scope.go:117] "RemoveContainer" containerID="c9c7a58ae9ce65a8142a0791b5158b28d7e0a2bdce7da5e01446cf7b2e6affac"
Jul 30 13:32:22.709617 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.709587    2557 scope.go:117] "RemoveContainer" containerID="8cdfe6ec44bef6bc788e5ccd541e32cb962266926398ba1ed621c7e23d3a3bcc"
Jul 30 13:32:22.709828 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:32:22.709811    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cdfe6ec44bef6bc788e5ccd541e32cb962266926398ba1ed621c7e23d3a3bcc\": container with ID starting with 8cdfe6ec44bef6bc788e5ccd541e32cb962266926398ba1ed621c7e23d3a3bcc not found: ID does not exist" containerID="8cdfe6ec44bef6bc788e5ccd541e32cb962266926398ba1ed621c7e23d3a3bcc"
Jul 30 13:32:22.709875 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.709836    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cdfe6ec44bef6bc788e5ccd541e32cb962266926398ba1ed621c7e23d3a3bcc"} err="failed to get container status \"8cdfe6ec44bef6bc788e5ccd541e32cb962266926398ba1ed621c7e23d3a3bcc\": rpc error: code = NotFound desc = could not find container \"8cdfe6ec44bef6bc788e5ccd541e32cb962266926398ba1ed621c7e23d3a3bcc\": container with ID starting with 8cdfe6ec44bef6bc788e5ccd541e32cb962266926398ba1ed621c7e23d3a3bcc not found: ID does not exist"
Jul 30 13:32:22.709875 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.709852    2557 scope.go:117] "RemoveContainer" containerID="c9c7a58ae9ce65a8142a0791b5158b28d7e0a2bdce7da5e01446cf7b2e6affac"
Jul 30 13:32:22.710091 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:32:22.710074    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9c7a58ae9ce65a8142a0791b5158b28d7e0a2bdce7da5e01446cf7b2e6affac\": container with ID starting with c9c7a58ae9ce65a8142a0791b5158b28d7e0a2bdce7da5e01446cf7b2e6affac not found: ID does not exist" containerID="c9c7a58ae9ce65a8142a0791b5158b28d7e0a2bdce7da5e01446cf7b2e6affac"
Jul 30 13:32:22.710142 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.710095    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9c7a58ae9ce65a8142a0791b5158b28d7e0a2bdce7da5e01446cf7b2e6affac"} err="failed to get container status \"c9c7a58ae9ce65a8142a0791b5158b28d7e0a2bdce7da5e01446cf7b2e6affac\": rpc error: code = NotFound desc = could not find container \"c9c7a58ae9ce65a8142a0791b5158b28d7e0a2bdce7da5e01446cf7b2e6affac\": container with ID starting with c9c7a58ae9ce65a8142a0791b5158b28d7e0a2bdce7da5e01446cf7b2e6affac not found: ID does not exist"
Jul 30 13:32:22.716413 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.716395    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-istio-envoy\") pod \"9569c782-fd52-488e-81a5-27560c649ef1\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") "
Jul 30 13:32:22.716554 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.716426    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9569c782-fd52-488e-81a5-27560c649ef1-istio-token\") pod \"9569c782-fd52-488e-81a5-27560c649ef1\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") "
Jul 30 13:32:22.716554 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.716458    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-credential-socket\") pod \"9569c782-fd52-488e-81a5-27560c649ef1\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") "
Jul 30 13:32:22.716554 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.716475    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-workload-certs\") pod \"9569c782-fd52-488e-81a5-27560c649ef1\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") "
Jul 30 13:32:22.716554 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.716504    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-workload-socket\") pod \"9569c782-fd52-488e-81a5-27560c649ef1\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") "
Jul 30 13:32:22.716807 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.716552    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-istio-data\") pod \"9569c782-fd52-488e-81a5-27560c649ef1\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") "
Jul 30 13:32:22.716807 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.716605    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9569c782-fd52-488e-81a5-27560c649ef1-istiod-ca-cert\") pod \"9569c782-fd52-488e-81a5-27560c649ef1\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") "
Jul 30 13:32:22.716807 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.716648    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9569c782-fd52-488e-81a5-27560c649ef1-istio-podinfo\") pod \"9569c782-fd52-488e-81a5-27560c649ef1\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") "
Jul 30 13:32:22.716807 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.716680    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs8mb\" (UniqueName: \"kubernetes.io/projected/9569c782-fd52-488e-81a5-27560c649ef1-kube-api-access-zs8mb\") pod \"9569c782-fd52-488e-81a5-27560c649ef1\" (UID: \"9569c782-fd52-488e-81a5-27560c649ef1\") "
Jul 30 13:32:22.716807 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.716762    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9569c782-fd52-488e-81a5-27560c649ef1" (UID: "9569c782-fd52-488e-81a5-27560c649ef1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:32:22.717185 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.716993    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-credential-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:22.717185 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.717056    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9569c782-fd52-488e-81a5-27560c649ef1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9569c782-fd52-488e-81a5-27560c649ef1" (UID: "9569c782-fd52-488e-81a5-27560c649ef1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:32:22.717477 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.717450    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9569c782-fd52-488e-81a5-27560c649ef1" (UID: "9569c782-fd52-488e-81a5-27560c649ef1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:32:22.717573 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.717515    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9569c782-fd52-488e-81a5-27560c649ef1" (UID: "9569c782-fd52-488e-81a5-27560c649ef1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:32:22.717873 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.717852    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-istio-data" (OuterVolumeSpecName: "istio-data") pod "9569c782-fd52-488e-81a5-27560c649ef1" (UID: "9569c782-fd52-488e-81a5-27560c649ef1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:32:22.718710 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.718685    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9569c782-fd52-488e-81a5-27560c649ef1-istio-token" (OuterVolumeSpecName: "istio-token") pod "9569c782-fd52-488e-81a5-27560c649ef1" (UID: "9569c782-fd52-488e-81a5-27560c649ef1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:32:22.718858 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.718839    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9569c782-fd52-488e-81a5-27560c649ef1" (UID: "9569c782-fd52-488e-81a5-27560c649ef1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:32:22.718938 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.718886    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9569c782-fd52-488e-81a5-27560c649ef1-kube-api-access-zs8mb" (OuterVolumeSpecName: "kube-api-access-zs8mb") pod "9569c782-fd52-488e-81a5-27560c649ef1" (UID: "9569c782-fd52-488e-81a5-27560c649ef1"). InnerVolumeSpecName "kube-api-access-zs8mb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:32:22.719251 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.719224    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9569c782-fd52-488e-81a5-27560c649ef1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9569c782-fd52-488e-81a5-27560c649ef1" (UID: "9569c782-fd52-488e-81a5-27560c649ef1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 13:32:22.817937 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.817906    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-istio-envoy\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:22.817937 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.817933    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9569c782-fd52-488e-81a5-27560c649ef1-istio-token\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:22.817937 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.817941    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-workload-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:22.818182 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.817949    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-workload-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:22.818182 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.817958    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9569c782-fd52-488e-81a5-27560c649ef1-istio-data\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:22.818182 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.817966    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9569c782-fd52-488e-81a5-27560c649ef1-istiod-ca-cert\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:22.818182 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.817974    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9569c782-fd52-488e-81a5-27560c649ef1-istio-podinfo\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:22.818182 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:22.817982    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs8mb\" (UniqueName: \"kubernetes.io/projected/9569c782-fd52-488e-81a5-27560c649ef1-kube-api-access-zs8mb\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:23.030954 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:23.030912    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"]
Jul 30 13:32:23.037978 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:23.037945    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-c5e6a00f-gw-openshift-default-5df45b776-m8sz9"]
Jul 30 13:32:23.184574 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:23.184534    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9569c782-fd52-488e-81a5-27560c649ef1" path="/var/lib/kubelet/pods/9569c782-fd52-488e-81a5-27560c649ef1/volumes"
Jul 30 13:32:31.931199 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:31.931115    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:32:35.682714 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:35.682675    2557 patch_prober.go:28] interesting pod/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 30 13:32:35.683214 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:35.682744    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw" podUID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 30 13:32:35.763715 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:35.763681    2557 generic.go:356] "Generic (PLEG): container finished" podID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerID="2b43c81cb9027d54537f8ad78dcbae5a98dd80b0831743bf64560c4ace6d7f61" exitCode=0
Jul 30 13:32:35.763882 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:35.763760    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw" event={"ID":"5a42b68f-9e61-4c25-87fa-b052a1435bc8","Type":"ContainerDied","Data":"2b43c81cb9027d54537f8ad78dcbae5a98dd80b0831743bf64560c4ace6d7f61"}
Jul 30 13:32:35.764180 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:35.764167    2557 scope.go:117] "RemoveContainer" containerID="2b43c81cb9027d54537f8ad78dcbae5a98dd80b0831743bf64560c4ace6d7f61"
Jul 30 13:32:35.765279 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:35.765255    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:32:35.765344 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:35.765334    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:32:35.765380 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:35.765366    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:32:36.774307 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:36.774264    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw" event={"ID":"5a42b68f-9e61-4c25-87fa-b052a1435bc8","Type":"ContainerStarted","Data":"1d077426ebcff152411997215089494273bd5c9b2e2a46fb9eb16e9c88b59d2c"}
Jul 30 13:32:37.600254 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:37.600132    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:37.600528 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:37.600502    2557 patch_prober.go:28] interesting pod/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 30 13:32:37.600667 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:37.600576    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw" podUID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 30 13:32:38.599411 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:38.599375    2557 patch_prober.go:28] interesting pod/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 30 13:32:38.599867 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:38.599428    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw" podUID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 30 13:32:39.599962 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:39.599927    2557 patch_prober.go:28] interesting pod/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 30 13:32:39.600363 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:39.599988    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw" podUID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 30 13:32:40.599840 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:40.599800    2557 patch_prober.go:28] interesting pod/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 30 13:32:40.600021 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:40.599855    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw" podUID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 30 13:32:41.603609 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:41.603565    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:41.603985 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:41.603928    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:41.604631 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:41.604616    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:46.922392 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:46.922359    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s"]
Jul 30 13:32:46.922826 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:46.922717    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9569c782-fd52-488e-81a5-27560c649ef1" containerName="istio-proxy"
Jul 30 13:32:46.922826 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:46.922729    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9569c782-fd52-488e-81a5-27560c649ef1" containerName="istio-proxy"
Jul 30 13:32:46.922826 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:46.922738    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9569c782-fd52-488e-81a5-27560c649ef1" containerName="istio-proxy"
Jul 30 13:32:46.922826 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:46.922744    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9569c782-fd52-488e-81a5-27560c649ef1" containerName="istio-proxy"
Jul 30 13:32:46.922826 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:46.922799    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9569c782-fd52-488e-81a5-27560c649ef1" containerName="istio-proxy"
Jul 30 13:32:46.925914 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:46.925895    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s"
Jul 30 13:32:46.928883 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:46.928865    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxdpz\""
Jul 30 13:32:46.928989 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:46.928927    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 13:32:46.934197 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:46.934174    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s"]
Jul 30 13:32:46.947796 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:46.947765    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ada707-bcb2-4cae-821d-6c05b79d64f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c625014c-stk2s\" (UID: \"74ada707-bcb2-4cae-821d-6c05b79d64f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s"
Jul 30 13:32:47.048359 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:47.048329    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ada707-bcb2-4cae-821d-6c05b79d64f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c625014c-stk2s\" (UID: \"74ada707-bcb2-4cae-821d-6c05b79d64f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s"
Jul 30 13:32:47.048936 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:47.048918    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ada707-bcb2-4cae-821d-6c05b79d64f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c625014c-stk2s\" (UID: \"74ada707-bcb2-4cae-821d-6c05b79d64f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s"
Jul 30 13:32:47.237639 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:47.237545    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s"
Jul 30 13:32:47.371411 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:47.371384    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s"]
Jul 30 13:32:47.373912 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:32:47.373873    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ada707_bcb2_4cae_821d_6c05b79d64f4.slice/crio-ea26e3efacd18fc8bee95542c4c3d4845d08b2547ccd7450d5b0544b7eec4221 WatchSource:0}: Error finding container ea26e3efacd18fc8bee95542c4c3d4845d08b2547ccd7450d5b0544b7eec4221: Status 404 returned error can't find the container with id ea26e3efacd18fc8bee95542c4c3d4845d08b2547ccd7450d5b0544b7eec4221
Jul 30 13:32:47.837360 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:47.837274    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s" event={"ID":"74ada707-bcb2-4cae-821d-6c05b79d64f4","Type":"ContainerStarted","Data":"1e67f96e218ac9d483e1216bad9a7f705378d1f51b30aabcf5faa0c86afac8ec"}
Jul 30 13:32:47.837360 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:47.837328    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s" event={"ID":"74ada707-bcb2-4cae-821d-6c05b79d64f4","Type":"ContainerStarted","Data":"ea26e3efacd18fc8bee95542c4c3d4845d08b2547ccd7450d5b0544b7eec4221"}
Jul 30 13:32:47.855727 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:47.855665    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s" podStartSLOduration=1.8556503420000001 podStartE2EDuration="1.855650342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:32:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:32:47.854531673 +0000 UTC m=+1991.126996072" watchObservedRunningTime="2026-07-30 13:32:47.855650342 +0000 UTC m=+1991.128114738"
Jul 30 13:32:48.842979 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:48.842946    2557 generic.go:356] "Generic (PLEG): container finished" podID="74ada707-bcb2-4cae-821d-6c05b79d64f4" containerID="1e67f96e218ac9d483e1216bad9a7f705378d1f51b30aabcf5faa0c86afac8ec" exitCode=0
Jul 30 13:32:48.843383 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:48.842994    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s" event={"ID":"74ada707-bcb2-4cae-821d-6c05b79d64f4","Type":"ContainerDied","Data":"1e67f96e218ac9d483e1216bad9a7f705378d1f51b30aabcf5faa0c86afac8ec"}
Jul 30 13:32:49.974972 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:49.974949    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s"
Jul 30 13:32:50.077768 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:50.077732    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ada707-bcb2-4cae-821d-6c05b79d64f4-maas-api-service-ca\") pod \"74ada707-bcb2-4cae-821d-6c05b79d64f4\" (UID: \"74ada707-bcb2-4cae-821d-6c05b79d64f4\") "
Jul 30 13:32:50.078054 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:50.078029    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74ada707-bcb2-4cae-821d-6c05b79d64f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "74ada707-bcb2-4cae-821d-6c05b79d64f4" (UID: "74ada707-bcb2-4cae-821d-6c05b79d64f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:32:50.178442 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:50.178415    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ada707-bcb2-4cae-821d-6c05b79d64f4-maas-api-service-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:50.853629 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:50.853586    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s"
Jul 30 13:32:50.853629 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:50.853621    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s" event={"ID":"74ada707-bcb2-4cae-821d-6c05b79d64f4","Type":"ContainerDied","Data":"ea26e3efacd18fc8bee95542c4c3d4845d08b2547ccd7450d5b0544b7eec4221"}
Jul 30 13:32:50.853839 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:50.853663    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea26e3efacd18fc8bee95542c4c3d4845d08b2547ccd7450d5b0544b7eec4221"
Jul 30 13:32:53.001106 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:53.001066    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s"]
Jul 30 13:32:53.002718 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:53.002693    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c625014c-stk2s"]
Jul 30 13:32:53.183139 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:53.183106    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74ada707-bcb2-4cae-821d-6c05b79d64f4" path="/var/lib/kubelet/pods/74ada707-bcb2-4cae-821d-6c05b79d64f4/volumes"
Jul 30 13:32:54.248772 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:54.248737    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"]
Jul 30 13:32:54.249302 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:54.248934    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw" podUID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerName="istio-proxy" containerID="cri-o://1d077426ebcff152411997215089494273bd5c9b2e2a46fb9eb16e9c88b59d2c" gracePeriod=30
Jul 30 13:32:55.006134 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.006099    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"]
Jul 30 13:32:55.006522 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.006505    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74ada707-bcb2-4cae-821d-6c05b79d64f4" containerName="revoke-keys"
Jul 30 13:32:55.006638 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.006525    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="74ada707-bcb2-4cae-821d-6c05b79d64f4" containerName="revoke-keys"
Jul 30 13:32:55.006707 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.006655    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9569c782-fd52-488e-81a5-27560c649ef1" containerName="istio-proxy"
Jul 30 13:32:55.006707 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.006674    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="74ada707-bcb2-4cae-821d-6c05b79d64f4" containerName="revoke-keys"
Jul 30 13:32:55.010673 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.010633    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.017738 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.017713    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-f10a044c-openshift-default-dockercfg-7bh9h\""
Jul 30 13:32:55.030513 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.030487    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"]
Jul 30 13:32:55.122376 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.122343    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-credential-socket\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.122376 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.122374    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/872a82af-f437-47c6-bf98-f34f37a0a654-istio-token\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.122623 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.122396    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-workload-certs\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.122623 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.122411    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/872a82af-f437-47c6-bf98-f34f37a0a654-istio-podinfo\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.122623 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.122437    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/872a82af-f437-47c6-bf98-f34f37a0a654-istiod-ca-cert\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.122623 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.122520    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-istio-data\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.122623 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.122547    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-workload-socket\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.122623 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.122567    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-istio-envoy\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.122623 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.122582    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-427jz\" (UniqueName: \"kubernetes.io/projected/872a82af-f437-47c6-bf98-f34f37a0a654-kube-api-access-427jz\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.224008 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.223978    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-credential-socket\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.224008 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.224010    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/872a82af-f437-47c6-bf98-f34f37a0a654-istio-token\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.224251 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.224034    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-workload-certs\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.224251 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.224074    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/872a82af-f437-47c6-bf98-f34f37a0a654-istio-podinfo\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.224251 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.224122    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/872a82af-f437-47c6-bf98-f34f37a0a654-istiod-ca-cert\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.224251 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.224171    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-istio-data\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.224251 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.224200    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-workload-socket\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.224557 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.224350    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-istio-envoy\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.224557 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.224396    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-427jz\" (UniqueName: \"kubernetes.io/projected/872a82af-f437-47c6-bf98-f34f37a0a654-kube-api-access-427jz\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.224557 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.224403    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-credential-socket\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.224557 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.224501    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-workload-socket\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.224781 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.224640    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-istio-data\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.224781 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.224704    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-workload-certs\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.224967 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.224940    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/872a82af-f437-47c6-bf98-f34f37a0a654-istiod-ca-cert\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.226436 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.226415    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-istio-envoy\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.226743 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.226727    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/872a82af-f437-47c6-bf98-f34f37a0a654-istio-podinfo\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.238150 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.238124    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/872a82af-f437-47c6-bf98-f34f37a0a654-istio-token\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.239062 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.239040    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-427jz\" (UniqueName: \"kubernetes.io/projected/872a82af-f437-47c6-bf98-f34f37a0a654-kube-api-access-427jz\") pod \"e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") " pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.323337 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.323246    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:55.458251 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.458215    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"]
Jul 30 13:32:55.461231 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:32:55.461205    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod872a82af_f437_47c6_bf98_f34f37a0a654.slice/crio-b0031fe2a8f911a3e07cf42dc4e8bc62b0515dfdc01719e7babb2a6e849f57b2 WatchSource:0}: Error finding container b0031fe2a8f911a3e07cf42dc4e8bc62b0515dfdc01719e7babb2a6e849f57b2: Status 404 returned error can't find the container with id b0031fe2a8f911a3e07cf42dc4e8bc62b0515dfdc01719e7babb2a6e849f57b2
Jul 30 13:32:55.463183 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.463152    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:32:55.463268 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.463217    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:32:55.463268 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.463246    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:32:55.881346 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.881305    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x" event={"ID":"872a82af-f437-47c6-bf98-f34f37a0a654","Type":"ContainerStarted","Data":"d71d68a7bd1131dba66bb023290ce3d0f2832a55dd713e98253f2e76df00eb00"}
Jul 30 13:32:55.881546 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.881367    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x" event={"ID":"872a82af-f437-47c6-bf98-f34f37a0a654","Type":"ContainerStarted","Data":"b0031fe2a8f911a3e07cf42dc4e8bc62b0515dfdc01719e7babb2a6e849f57b2"}
Jul 30 13:32:55.909500 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:55.909450    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x" podStartSLOduration=1.909434921 podStartE2EDuration="1.909434921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:32:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:32:55.906858032 +0000 UTC m=+1999.179322430" watchObservedRunningTime="2026-07-30 13:32:55.909434921 +0000 UTC m=+1999.181899317"
Jul 30 13:32:56.324191 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:56.324152    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:56.328718 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:56.328692    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:56.606559 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:56.606471    2557 patch_prober.go:28] interesting pod/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 13:32:56.606559 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:56.606527    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw" podUID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 13:32:56.886425 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:56.886345    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:56.887232 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:56.887211    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:32:57.039974 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:57.039947    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:32:59.507118 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.507092    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:59.567881 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.567804    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5gqg\" (UniqueName: \"kubernetes.io/projected/5a42b68f-9e61-4c25-87fa-b052a1435bc8-kube-api-access-n5gqg\") pod \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") "
Jul 30 13:32:59.567881 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.567853    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-data\") pod \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") "
Jul 30 13:32:59.568089 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.567893    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-workload-socket\") pod \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") "
Jul 30 13:32:59.568089 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.567909    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-podinfo\") pod \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") "
Jul 30 13:32:59.568089 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.567925    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istiod-ca-cert\") pod \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") "
Jul 30 13:32:59.568089 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.567942    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-credential-socket\") pod \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") "
Jul 30 13:32:59.568089 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.567969    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-token\") pod \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") "
Jul 30 13:32:59.568326 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.568143    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-envoy\") pod \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") "
Jul 30 13:32:59.568326 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.568151    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5a42b68f-9e61-4c25-87fa-b052a1435bc8" (UID: "5a42b68f-9e61-4c25-87fa-b052a1435bc8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:32:59.568326 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.568183    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-workload-certs\") pod \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\" (UID: \"5a42b68f-9e61-4c25-87fa-b052a1435bc8\") "
Jul 30 13:32:59.568521 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.568502    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-workload-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:59.568952 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.568908    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5a42b68f-9e61-4c25-87fa-b052a1435bc8" (UID: "5a42b68f-9e61-4c25-87fa-b052a1435bc8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:32:59.568952 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.568933    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5a42b68f-9e61-4c25-87fa-b052a1435bc8" (UID: "5a42b68f-9e61-4c25-87fa-b052a1435bc8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:32:59.569166 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.569043    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5a42b68f-9e61-4c25-87fa-b052a1435bc8" (UID: "5a42b68f-9e61-4c25-87fa-b052a1435bc8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:32:59.569374 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.569352    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-data" (OuterVolumeSpecName: "istio-data") pod "5a42b68f-9e61-4c25-87fa-b052a1435bc8" (UID: "5a42b68f-9e61-4c25-87fa-b052a1435bc8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:32:59.570341 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.570316    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a42b68f-9e61-4c25-87fa-b052a1435bc8-kube-api-access-n5gqg" (OuterVolumeSpecName: "kube-api-access-n5gqg") pod "5a42b68f-9e61-4c25-87fa-b052a1435bc8" (UID: "5a42b68f-9e61-4c25-87fa-b052a1435bc8"). InnerVolumeSpecName "kube-api-access-n5gqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:32:59.570821 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.570801    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5a42b68f-9e61-4c25-87fa-b052a1435bc8" (UID: "5a42b68f-9e61-4c25-87fa-b052a1435bc8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 13:32:59.570998 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.570972    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-token" (OuterVolumeSpecName: "istio-token") pod "5a42b68f-9e61-4c25-87fa-b052a1435bc8" (UID: "5a42b68f-9e61-4c25-87fa-b052a1435bc8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:32:59.571084 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.571034    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5a42b68f-9e61-4c25-87fa-b052a1435bc8" (UID: "5a42b68f-9e61-4c25-87fa-b052a1435bc8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:32:59.670003 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.669967    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5gqg\" (UniqueName: \"kubernetes.io/projected/5a42b68f-9e61-4c25-87fa-b052a1435bc8-kube-api-access-n5gqg\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:59.670003 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.669996    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-data\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:59.670003 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.670005    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-podinfo\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:59.670003 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.670013    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istiod-ca-cert\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:59.670265 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.670022    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-credential-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:59.670265 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.670030    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-token\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:59.670265 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.670039    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-istio-envoy\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:59.670265 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.670046    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a42b68f-9e61-4c25-87fa-b052a1435bc8-workload-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:32:59.902951 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.902857    2557 generic.go:356] "Generic (PLEG): container finished" podID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerID="1d077426ebcff152411997215089494273bd5c9b2e2a46fb9eb16e9c88b59d2c" exitCode=0
Jul 30 13:32:59.902951 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.902914    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"
Jul 30 13:32:59.903133 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.902946    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw" event={"ID":"5a42b68f-9e61-4c25-87fa-b052a1435bc8","Type":"ContainerDied","Data":"1d077426ebcff152411997215089494273bd5c9b2e2a46fb9eb16e9c88b59d2c"}
Jul 30 13:32:59.903133 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.903031    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw" event={"ID":"5a42b68f-9e61-4c25-87fa-b052a1435bc8","Type":"ContainerDied","Data":"512b89e9e162b340ddd600e8dc7c34312e9d0c673d1f1d09db27dd355bc48202"}
Jul 30 13:32:59.903133 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.903056    2557 scope.go:117] "RemoveContainer" containerID="1d077426ebcff152411997215089494273bd5c9b2e2a46fb9eb16e9c88b59d2c"
Jul 30 13:32:59.913073 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.913057    2557 scope.go:117] "RemoveContainer" containerID="2b43c81cb9027d54537f8ad78dcbae5a98dd80b0831743bf64560c4ace6d7f61"
Jul 30 13:32:59.920489 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.920472    2557 scope.go:117] "RemoveContainer" containerID="1d077426ebcff152411997215089494273bd5c9b2e2a46fb9eb16e9c88b59d2c"
Jul 30 13:32:59.920745 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:32:59.920727    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d077426ebcff152411997215089494273bd5c9b2e2a46fb9eb16e9c88b59d2c\": container with ID starting with 1d077426ebcff152411997215089494273bd5c9b2e2a46fb9eb16e9c88b59d2c not found: ID does not exist" containerID="1d077426ebcff152411997215089494273bd5c9b2e2a46fb9eb16e9c88b59d2c"
Jul 30 13:32:59.920798 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.920754    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d077426ebcff152411997215089494273bd5c9b2e2a46fb9eb16e9c88b59d2c"} err="failed to get container status \"1d077426ebcff152411997215089494273bd5c9b2e2a46fb9eb16e9c88b59d2c\": rpc error: code = NotFound desc = could not find container \"1d077426ebcff152411997215089494273bd5c9b2e2a46fb9eb16e9c88b59d2c\": container with ID starting with 1d077426ebcff152411997215089494273bd5c9b2e2a46fb9eb16e9c88b59d2c not found: ID does not exist"
Jul 30 13:32:59.920798 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.920771    2557 scope.go:117] "RemoveContainer" containerID="2b43c81cb9027d54537f8ad78dcbae5a98dd80b0831743bf64560c4ace6d7f61"
Jul 30 13:32:59.921020 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:32:59.921003    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b43c81cb9027d54537f8ad78dcbae5a98dd80b0831743bf64560c4ace6d7f61\": container with ID starting with 2b43c81cb9027d54537f8ad78dcbae5a98dd80b0831743bf64560c4ace6d7f61 not found: ID does not exist" containerID="2b43c81cb9027d54537f8ad78dcbae5a98dd80b0831743bf64560c4ace6d7f61"
Jul 30 13:32:59.921062 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.921026    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b43c81cb9027d54537f8ad78dcbae5a98dd80b0831743bf64560c4ace6d7f61"} err="failed to get container status \"2b43c81cb9027d54537f8ad78dcbae5a98dd80b0831743bf64560c4ace6d7f61\": rpc error: code = NotFound desc = could not find container \"2b43c81cb9027d54537f8ad78dcbae5a98dd80b0831743bf64560c4ace6d7f61\": container with ID starting with 2b43c81cb9027d54537f8ad78dcbae5a98dd80b0831743bf64560c4ace6d7f61 not found: ID does not exist"
Jul 30 13:32:59.937720 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.937693    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"]
Jul 30 13:32:59.954841 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:32:59.954814    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c625014c-openshift-default-66b5b57976-hq8nw"]
Jul 30 13:33:01.183401 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:01.183369    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" path="/var/lib/kubelet/pods/5a42b68f-9e61-4c25-87fa-b052a1435bc8/volumes"
Jul 30 13:33:07.828113 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:07.828078    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:33:11.888167 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:11.888082    2557 patch_prober.go:28] interesting pod/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 30 13:33:11.888167 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:11.888147    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x" podUID="872a82af-f437-47c6-bf98-f34f37a0a654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 30 13:33:11.963146 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:11.963114    2557 generic.go:356] "Generic (PLEG): container finished" podID="872a82af-f437-47c6-bf98-f34f37a0a654" containerID="d71d68a7bd1131dba66bb023290ce3d0f2832a55dd713e98253f2e76df00eb00" exitCode=0
Jul 30 13:33:11.963297 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:11.963183    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x" event={"ID":"872a82af-f437-47c6-bf98-f34f37a0a654","Type":"ContainerDied","Data":"d71d68a7bd1131dba66bb023290ce3d0f2832a55dd713e98253f2e76df00eb00"}
Jul 30 13:33:11.963568 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:11.963553    2557 scope.go:117] "RemoveContainer" containerID="d71d68a7bd1131dba66bb023290ce3d0f2832a55dd713e98253f2e76df00eb00"
Jul 30 13:33:11.964494 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:11.964460    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:33:11.964545 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:11.964530    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:33:11.964579 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:11.964561    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:33:12.973121 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:12.973076    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x" event={"ID":"872a82af-f437-47c6-bf98-f34f37a0a654","Type":"ContainerStarted","Data":"8dfa9268277ceec62af20bb3c06342b4230a5ff7b5cc06fe0b18192172e562d1"}
Jul 30 13:33:13.324332 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:13.324249    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:33:13.324475 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:13.324422    2557 patch_prober.go:28] interesting pod/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 30 13:33:13.324516 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:13.324469    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x" podUID="872a82af-f437-47c6-bf98-f34f37a0a654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 30 13:33:14.323962 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:14.323922    2557 patch_prober.go:28] interesting pod/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 30 13:33:14.324351 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:14.323980    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x" podUID="872a82af-f437-47c6-bf98-f34f37a0a654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 30 13:33:15.323987 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:15.323952    2557 patch_prober.go:28] interesting pod/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 30 13:33:15.324391 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:15.324002    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x" podUID="872a82af-f437-47c6-bf98-f34f37a0a654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 30 13:33:16.324564 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:16.324528    2557 patch_prober.go:28] interesting pod/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 30 13:33:16.325006 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:16.324580    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x" podUID="872a82af-f437-47c6-bf98-f34f37a0a654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 30 13:33:17.327833 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:17.327802    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:33:17.328298 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:17.328199    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:33:17.329034 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:17.329016    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:33:22.185214 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.185171    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45"]
Jul 30 13:33:22.185764 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.185746    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerName="istio-proxy"
Jul 30 13:33:22.185821 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.185768    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerName="istio-proxy"
Jul 30 13:33:22.185821 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.185786    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerName="istio-proxy"
Jul 30 13:33:22.185821 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.185795    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerName="istio-proxy"
Jul 30 13:33:22.186101 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.185887    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerName="istio-proxy"
Jul 30 13:33:22.186101 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.185897    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a42b68f-9e61-4c25-87fa-b052a1435bc8" containerName="istio-proxy"
Jul 30 13:33:22.188233 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.188215    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45"
Jul 30 13:33:22.191379 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.191349    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxdpz\""
Jul 30 13:33:22.191518 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.191379    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 13:33:22.195716 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.195690    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45"]
Jul 30 13:33:22.277567 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.277530    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3bdc78f-5a37-4840-a3af-3e21e3c8634d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f10a044c-jwd45\" (UID: \"e3bdc78f-5a37-4840-a3af-3e21e3c8634d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45"
Jul 30 13:33:22.378362 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.378321    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3bdc78f-5a37-4840-a3af-3e21e3c8634d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f10a044c-jwd45\" (UID: \"e3bdc78f-5a37-4840-a3af-3e21e3c8634d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45"
Jul 30 13:33:22.379022 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.378998    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3bdc78f-5a37-4840-a3af-3e21e3c8634d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f10a044c-jwd45\" (UID: \"e3bdc78f-5a37-4840-a3af-3e21e3c8634d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45"
Jul 30 13:33:22.500841 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.500759    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45"
Jul 30 13:33:22.662407 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:22.662388    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45"]
Jul 30 13:33:22.665126 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:33:22.665100    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3bdc78f_5a37_4840_a3af_3e21e3c8634d.slice/crio-f4ac30a5e9a52264baf7a0449f3cb9ade42399d300a08322ac843b43726b78f3 WatchSource:0}: Error finding container f4ac30a5e9a52264baf7a0449f3cb9ade42399d300a08322ac843b43726b78f3: Status 404 returned error can't find the container with id f4ac30a5e9a52264baf7a0449f3cb9ade42399d300a08322ac843b43726b78f3
Jul 30 13:33:23.027326 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:23.027281    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45" event={"ID":"e3bdc78f-5a37-4840-a3af-3e21e3c8634d","Type":"ContainerStarted","Data":"e5898e444f378ec6886dbcfd04735631d7fd80df0157ffdd339cda8fe0948cfb"}
Jul 30 13:33:23.027499 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:23.027361    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45" event={"ID":"e3bdc78f-5a37-4840-a3af-3e21e3c8634d","Type":"ContainerStarted","Data":"f4ac30a5e9a52264baf7a0449f3cb9ade42399d300a08322ac843b43726b78f3"}
Jul 30 13:33:23.043984 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:23.043888    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45" podStartSLOduration=1.043872898 podStartE2EDuration="1.043872898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:33:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:33:23.042681309 +0000 UTC m=+2026.315145726" watchObservedRunningTime="2026-07-30 13:33:23.043872898 +0000 UTC m=+2026.316337295"
Jul 30 13:33:24.033612 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:24.033565    2557 generic.go:356] "Generic (PLEG): container finished" podID="e3bdc78f-5a37-4840-a3af-3e21e3c8634d" containerID="e5898e444f378ec6886dbcfd04735631d7fd80df0157ffdd339cda8fe0948cfb" exitCode=0
Jul 30 13:33:24.034057 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:24.033646    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45" event={"ID":"e3bdc78f-5a37-4840-a3af-3e21e3c8634d","Type":"ContainerDied","Data":"e5898e444f378ec6886dbcfd04735631d7fd80df0157ffdd339cda8fe0948cfb"}
Jul 30 13:33:25.165410 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:25.165388    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45"
Jul 30 13:33:25.303822 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:25.303736    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3bdc78f-5a37-4840-a3af-3e21e3c8634d-maas-api-service-ca\") pod \"e3bdc78f-5a37-4840-a3af-3e21e3c8634d\" (UID: \"e3bdc78f-5a37-4840-a3af-3e21e3c8634d\") "
Jul 30 13:33:25.304098 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:25.304075    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3bdc78f-5a37-4840-a3af-3e21e3c8634d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e3bdc78f-5a37-4840-a3af-3e21e3c8634d" (UID: "e3bdc78f-5a37-4840-a3af-3e21e3c8634d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:33:25.405056 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:25.405020    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3bdc78f-5a37-4840-a3af-3e21e3c8634d-maas-api-service-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:33:26.044509 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:26.044479    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45"
Jul 30 13:33:26.044707 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:26.044483    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45" event={"ID":"e3bdc78f-5a37-4840-a3af-3e21e3c8634d","Type":"ContainerDied","Data":"f4ac30a5e9a52264baf7a0449f3cb9ade42399d300a08322ac843b43726b78f3"}
Jul 30 13:33:26.044707 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:26.044619    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4ac30a5e9a52264baf7a0449f3cb9ade42399d300a08322ac843b43726b78f3"
Jul 30 13:33:28.981323 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:28.981285    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45"]
Jul 30 13:33:28.984573 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:28.984548    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f10a044c-jwd45"]
Jul 30 13:33:29.123052 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:29.123020    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"]
Jul 30 13:33:29.123290 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:29.123267    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x" podUID="872a82af-f437-47c6-bf98-f34f37a0a654" containerName="istio-proxy" containerID="cri-o://8dfa9268277ceec62af20bb3c06342b4230a5ff7b5cc06fe0b18192172e562d1" gracePeriod=30
Jul 30 13:33:29.183027 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:29.182994    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3bdc78f-5a37-4840-a3af-3e21e3c8634d" path="/var/lib/kubelet/pods/e3bdc78f-5a37-4840-a3af-3e21e3c8634d/volumes"
Jul 30 13:33:32.330591 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:32.330555    2557 patch_prober.go:28] interesting pod/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 13:33:32.330970 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:32.330621    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x" podUID="872a82af-f437-47c6-bf98-f34f37a0a654" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 13:33:32.741655 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:32.741623    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:33:34.376766 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.376745    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:33:34.486080 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486037    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-workload-socket\") pod \"872a82af-f437-47c6-bf98-f34f37a0a654\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") "
Jul 30 13:33:34.486274 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486142    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-istio-data\") pod \"872a82af-f437-47c6-bf98-f34f37a0a654\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") "
Jul 30 13:33:34.486274 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486163    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-istio-envoy\") pod \"872a82af-f437-47c6-bf98-f34f37a0a654\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") "
Jul 30 13:33:34.486274 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486186    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-credential-socket\") pod \"872a82af-f437-47c6-bf98-f34f37a0a654\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") "
Jul 30 13:33:34.486274 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486209    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/872a82af-f437-47c6-bf98-f34f37a0a654-istio-token\") pod \"872a82af-f437-47c6-bf98-f34f37a0a654\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") "
Jul 30 13:33:34.486274 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486239    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/872a82af-f437-47c6-bf98-f34f37a0a654-istiod-ca-cert\") pod \"872a82af-f437-47c6-bf98-f34f37a0a654\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") "
Jul 30 13:33:34.486274 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486274    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-workload-certs\") pod \"872a82af-f437-47c6-bf98-f34f37a0a654\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") "
Jul 30 13:33:34.486566 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486274    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "872a82af-f437-47c6-bf98-f34f37a0a654" (UID: "872a82af-f437-47c6-bf98-f34f37a0a654"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:33:34.486566 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486302    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/872a82af-f437-47c6-bf98-f34f37a0a654-istio-podinfo\") pod \"872a82af-f437-47c6-bf98-f34f37a0a654\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") "
Jul 30 13:33:34.486566 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486507    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-427jz\" (UniqueName: \"kubernetes.io/projected/872a82af-f437-47c6-bf98-f34f37a0a654-kube-api-access-427jz\") pod \"872a82af-f437-47c6-bf98-f34f37a0a654\" (UID: \"872a82af-f437-47c6-bf98-f34f37a0a654\") "
Jul 30 13:33:34.486741 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486543    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "872a82af-f437-47c6-bf98-f34f37a0a654" (UID: "872a82af-f437-47c6-bf98-f34f37a0a654"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:33:34.486741 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486618    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "872a82af-f437-47c6-bf98-f34f37a0a654" (UID: "872a82af-f437-47c6-bf98-f34f37a0a654"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:33:34.486741 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486696    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/872a82af-f437-47c6-bf98-f34f37a0a654-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "872a82af-f437-47c6-bf98-f34f37a0a654" (UID: "872a82af-f437-47c6-bf98-f34f37a0a654"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:33:34.486904 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486869    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-credential-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:33:34.486904 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486897    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/872a82af-f437-47c6-bf98-f34f37a0a654-istiod-ca-cert\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:33:34.486988 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486911    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-workload-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:33:34.486988 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.486924    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-workload-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:33:34.487246 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.487222    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-istio-data" (OuterVolumeSpecName: "istio-data") pod "872a82af-f437-47c6-bf98-f34f37a0a654" (UID: "872a82af-f437-47c6-bf98-f34f37a0a654"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:33:34.488706 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.488673    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/872a82af-f437-47c6-bf98-f34f37a0a654-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "872a82af-f437-47c6-bf98-f34f37a0a654" (UID: "872a82af-f437-47c6-bf98-f34f37a0a654"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 13:33:34.488827 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.488716    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/872a82af-f437-47c6-bf98-f34f37a0a654-istio-token" (OuterVolumeSpecName: "istio-token") pod "872a82af-f437-47c6-bf98-f34f37a0a654" (UID: "872a82af-f437-47c6-bf98-f34f37a0a654"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:33:34.488917 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.488900    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/872a82af-f437-47c6-bf98-f34f37a0a654-kube-api-access-427jz" (OuterVolumeSpecName: "kube-api-access-427jz") pod "872a82af-f437-47c6-bf98-f34f37a0a654" (UID: "872a82af-f437-47c6-bf98-f34f37a0a654"). InnerVolumeSpecName "kube-api-access-427jz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:33:34.488990 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.488954    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "872a82af-f437-47c6-bf98-f34f37a0a654" (UID: "872a82af-f437-47c6-bf98-f34f37a0a654"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:33:34.587934 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.587892    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-istio-data\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:33:34.587934 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.587928    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/872a82af-f437-47c6-bf98-f34f37a0a654-istio-envoy\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:33:34.587934 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.587937    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/872a82af-f437-47c6-bf98-f34f37a0a654-istio-token\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:33:34.587934 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.587946    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/872a82af-f437-47c6-bf98-f34f37a0a654-istio-podinfo\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:33:34.587934 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:34.587954    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-427jz\" (UniqueName: \"kubernetes.io/projected/872a82af-f437-47c6-bf98-f34f37a0a654-kube-api-access-427jz\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:33:35.088071 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:35.088032    2557 generic.go:356] "Generic (PLEG): container finished" podID="872a82af-f437-47c6-bf98-f34f37a0a654" containerID="8dfa9268277ceec62af20bb3c06342b4230a5ff7b5cc06fe0b18192172e562d1" exitCode=0
Jul 30 13:33:35.088237 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:35.088090    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"
Jul 30 13:33:35.088237 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:35.088121    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x" event={"ID":"872a82af-f437-47c6-bf98-f34f37a0a654","Type":"ContainerDied","Data":"8dfa9268277ceec62af20bb3c06342b4230a5ff7b5cc06fe0b18192172e562d1"}
Jul 30 13:33:35.088237 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:35.088202    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x" event={"ID":"872a82af-f437-47c6-bf98-f34f37a0a654","Type":"ContainerDied","Data":"b0031fe2a8f911a3e07cf42dc4e8bc62b0515dfdc01719e7babb2a6e849f57b2"}
Jul 30 13:33:35.088237 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:35.088225    2557 scope.go:117] "RemoveContainer" containerID="8dfa9268277ceec62af20bb3c06342b4230a5ff7b5cc06fe0b18192172e562d1"
Jul 30 13:33:35.097458 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:35.097441    2557 scope.go:117] "RemoveContainer" containerID="d71d68a7bd1131dba66bb023290ce3d0f2832a55dd713e98253f2e76df00eb00"
Jul 30 13:33:35.105385 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:35.105367    2557 scope.go:117] "RemoveContainer" containerID="8dfa9268277ceec62af20bb3c06342b4230a5ff7b5cc06fe0b18192172e562d1"
Jul 30 13:33:35.105656 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:33:35.105636    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dfa9268277ceec62af20bb3c06342b4230a5ff7b5cc06fe0b18192172e562d1\": container with ID starting with 8dfa9268277ceec62af20bb3c06342b4230a5ff7b5cc06fe0b18192172e562d1 not found: ID does not exist" containerID="8dfa9268277ceec62af20bb3c06342b4230a5ff7b5cc06fe0b18192172e562d1"
Jul 30 13:33:35.105716 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:35.105665    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dfa9268277ceec62af20bb3c06342b4230a5ff7b5cc06fe0b18192172e562d1"} err="failed to get container status \"8dfa9268277ceec62af20bb3c06342b4230a5ff7b5cc06fe0b18192172e562d1\": rpc error: code = NotFound desc = could not find container \"8dfa9268277ceec62af20bb3c06342b4230a5ff7b5cc06fe0b18192172e562d1\": container with ID starting with 8dfa9268277ceec62af20bb3c06342b4230a5ff7b5cc06fe0b18192172e562d1 not found: ID does not exist"
Jul 30 13:33:35.105716 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:35.105682    2557 scope.go:117] "RemoveContainer" containerID="d71d68a7bd1131dba66bb023290ce3d0f2832a55dd713e98253f2e76df00eb00"
Jul 30 13:33:35.105889 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:33:35.105871    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d71d68a7bd1131dba66bb023290ce3d0f2832a55dd713e98253f2e76df00eb00\": container with ID starting with d71d68a7bd1131dba66bb023290ce3d0f2832a55dd713e98253f2e76df00eb00 not found: ID does not exist" containerID="d71d68a7bd1131dba66bb023290ce3d0f2832a55dd713e98253f2e76df00eb00"
Jul 30 13:33:35.105930 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:35.105895    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d71d68a7bd1131dba66bb023290ce3d0f2832a55dd713e98253f2e76df00eb00"} err="failed to get container status \"d71d68a7bd1131dba66bb023290ce3d0f2832a55dd713e98253f2e76df00eb00\": rpc error: code = NotFound desc = could not find container \"d71d68a7bd1131dba66bb023290ce3d0f2832a55dd713e98253f2e76df00eb00\": container with ID starting with d71d68a7bd1131dba66bb023290ce3d0f2832a55dd713e98253f2e76df00eb00 not found: ID does not exist"
Jul 30 13:33:35.114227 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:35.114201    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"]
Jul 30 13:33:35.119891 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:35.119871    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-f10a044c-openshift-default-696c45b5dd-q5r9x"]
Jul 30 13:33:35.183605 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:35.183557    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="872a82af-f437-47c6-bf98-f34f37a0a654" path="/var/lib/kubelet/pods/872a82af-f437-47c6-bf98-f34f37a0a654/volumes"
Jul 30 13:33:54.846410 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:33:54.846375    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:35:27.644372 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:27.644340    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:35:39.337540 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:39.337463    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:35:41.010440 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.010401    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"]
Jul 30 13:35:41.010865 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.010849    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="872a82af-f437-47c6-bf98-f34f37a0a654" containerName="istio-proxy"
Jul 30 13:35:41.010909 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.010867    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="872a82af-f437-47c6-bf98-f34f37a0a654" containerName="istio-proxy"
Jul 30 13:35:41.010909 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.010879    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3bdc78f-5a37-4840-a3af-3e21e3c8634d" containerName="revoke-keys"
Jul 30 13:35:41.010909 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.010885    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3bdc78f-5a37-4840-a3af-3e21e3c8634d" containerName="revoke-keys"
Jul 30 13:35:41.011005 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.010957    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="872a82af-f437-47c6-bf98-f34f37a0a654" containerName="istio-proxy"
Jul 30 13:35:41.011005 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.010966    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="872a82af-f437-47c6-bf98-f34f37a0a654" containerName="istio-proxy"
Jul 30 13:35:41.011005 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.010975    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3bdc78f-5a37-4840-a3af-3e21e3c8634d" containerName="revoke-keys"
Jul 30 13:35:41.013969 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.013950    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.017775 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.017754    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-06db6c7f-openshift-default-dockercfg-nrqm4\""
Jul 30 13:35:41.018336 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.018323    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 13:35:41.045621 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.045585    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"]
Jul 30 13:35:41.061911 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.061874    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-workload-socket\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.062055 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.061934    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-data\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.062055 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.062007    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-credential-socket\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.062180 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.062072    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7qvz\" (UniqueName: \"kubernetes.io/projected/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-kube-api-access-n7qvz\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.062180 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.062117    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-workload-certs\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.062180 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.062141    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-envoy\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.062180 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.062168    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istiod-ca-cert\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.062360 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.062214    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-podinfo\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.062360 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.062253    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-token\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.163439 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.163402    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7qvz\" (UniqueName: \"kubernetes.io/projected/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-kube-api-access-n7qvz\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.163439 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.163446    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-workload-certs\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.163733 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.163467    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-envoy\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.163733 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.163484    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istiod-ca-cert\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.163733 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.163507    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-podinfo\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.163733 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.163545    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-token\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.163733 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.163577    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-workload-socket\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.163733 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.163655    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-data\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.163733 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.163683    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-credential-socket\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.164094 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.163898    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-workload-certs\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.164094 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.163984    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-workload-socket\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.164094 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.164020    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-credential-socket\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.164094 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.164079    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-data\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.164359 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.164339    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istiod-ca-cert\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.165887 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.165866    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-envoy\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.166107 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.166086    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-podinfo\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.172580 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.172549    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-token\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.173056 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.173030    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7qvz\" (UniqueName: \"kubernetes.io/projected/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-kube-api-access-n7qvz\") pod \"e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") " pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.323906 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.323818    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:41.461755 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.461728    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"]
Jul 30 13:35:41.463214 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:35:41.463187    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1837d7d_e9b3_4b84_9986_0a1942b9d97d.slice/crio-b7248cbd76ec0e8f6196391fe67c7c1dfafde3815cc0e1362306626dbad9c132 WatchSource:0}: Error finding container b7248cbd76ec0e8f6196391fe67c7c1dfafde3815cc0e1362306626dbad9c132: Status 404 returned error can't find the container with id b7248cbd76ec0e8f6196391fe67c7c1dfafde3815cc0e1362306626dbad9c132
Jul 30 13:35:41.465435 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.465394    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:35:41.465527 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.465484    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:35:41.465566 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.465523    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:35:41.688733 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.688698    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" event={"ID":"a1837d7d-e9b3-4b84-9986-0a1942b9d97d","Type":"ContainerStarted","Data":"b9146718f29f3c1194beb718ff51345fe6e11d1ea4a4b47cf2decdd8d884ebb0"}
Jul 30 13:35:41.688893 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.688761    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" event={"ID":"a1837d7d-e9b3-4b84-9986-0a1942b9d97d","Type":"ContainerStarted","Data":"b7248cbd76ec0e8f6196391fe67c7c1dfafde3815cc0e1362306626dbad9c132"}
Jul 30 13:35:41.716350 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:41.716290    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" podStartSLOduration=1.7162676380000002 podStartE2EDuration="1.716267638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:35:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:35:41.715350736 +0000 UTC m=+2164.987815155" watchObservedRunningTime="2026-07-30 13:35:41.716267638 +0000 UTC m=+2164.988732038"
Jul 30 13:35:42.324528 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:42.324498    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:42.329365 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:42.329342    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:42.700090 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:42.700057    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:42.700909 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:42.700891    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:35:51.335973 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:51.335893    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:35:58.742814 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:58.742767    2557 patch_prober.go:28] interesting pod/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 13:35:58.743339 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:58.742841    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 13:35:59.780171 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:59.780138    2557 generic.go:356] "Generic (PLEG): container finished" podID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerID="b9146718f29f3c1194beb718ff51345fe6e11d1ea4a4b47cf2decdd8d884ebb0" exitCode=0
Jul 30 13:35:59.780586 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:59.780211    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" event={"ID":"a1837d7d-e9b3-4b84-9986-0a1942b9d97d","Type":"ContainerDied","Data":"b9146718f29f3c1194beb718ff51345fe6e11d1ea4a4b47cf2decdd8d884ebb0"}
Jul 30 13:35:59.780586 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:59.780564    2557 scope.go:117] "RemoveContainer" containerID="b9146718f29f3c1194beb718ff51345fe6e11d1ea4a4b47cf2decdd8d884ebb0"
Jul 30 13:35:59.781547 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:59.781520    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:35:59.781619 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:59.781589    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:35:59.781660 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:35:59.781650    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:36:00.786889 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:00.786854    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" event={"ID":"a1837d7d-e9b3-4b84-9986-0a1942b9d97d","Type":"ContainerStarted","Data":"ab304667c1c8a5ca6ed0b066aea7e9c92c283f634545cabfe9d62afe9263fb8c"}
Jul 30 13:36:01.324563 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:01.324529    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:36:01.324749 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:01.324712    2557 patch_prober.go:28] interesting pod/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 30 13:36:01.324800 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:01.324762    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 30 13:36:02.324872 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:02.324832    2557 patch_prober.go:28] interesting pod/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 30 13:36:02.325346 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:02.324897    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 30 13:36:03.324302 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:03.324268    2557 patch_prober.go:28] interesting pod/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 30 13:36:03.324472 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:03.324322    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 30 13:36:04.325113 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:04.325080    2557 patch_prober.go:28] interesting pod/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 30 13:36:04.325493 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:04.325133    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 30 13:36:05.324854 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:05.324814    2557 patch_prober.go:28] interesting pod/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 30 13:36:05.325030 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:05.324884    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 30 13:36:06.329124 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:06.329092    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:36:06.329574 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:06.329419    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:36:06.330392 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:06.330374    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:36:16.045297 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.045260    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2"]
Jul 30 13:36:16.045686 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.045619    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="872a82af-f437-47c6-bf98-f34f37a0a654" containerName="istio-proxy"
Jul 30 13:36:16.045686 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.045632    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="872a82af-f437-47c6-bf98-f34f37a0a654" containerName="istio-proxy"
Jul 30 13:36:16.049625 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.049408    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2"
Jul 30 13:36:16.054033 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.054008    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 13:36:16.054445 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.054419    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxdpz\""
Jul 30 13:36:16.058739 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.058712    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2"]
Jul 30 13:36:16.090697 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.090666    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c768d1f7-4843-4756-aff5-60c4c40df284-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2\" (UID: \"c768d1f7-4843-4756-aff5-60c4c40df284\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2"
Jul 30 13:36:16.191512 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.191478    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c768d1f7-4843-4756-aff5-60c4c40df284-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2\" (UID: \"c768d1f7-4843-4756-aff5-60c4c40df284\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2"
Jul 30 13:36:16.192153 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.192131    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c768d1f7-4843-4756-aff5-60c4c40df284-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2\" (UID: \"c768d1f7-4843-4756-aff5-60c4c40df284\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2"
Jul 30 13:36:16.364236 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.364153    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2"
Jul 30 13:36:16.497991 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.497814    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2"]
Jul 30 13:36:16.500082 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:36:16.500046    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc768d1f7_4843_4756_aff5_60c4c40df284.slice/crio-7a22893bcf74064196b79c1cb1a9730fc1f9dfd766f1abec6405b604f480bc4d WatchSource:0}: Error finding container 7a22893bcf74064196b79c1cb1a9730fc1f9dfd766f1abec6405b604f480bc4d: Status 404 returned error can't find the container with id 7a22893bcf74064196b79c1cb1a9730fc1f9dfd766f1abec6405b604f480bc4d
Jul 30 13:36:16.864007 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.863975    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2" event={"ID":"c768d1f7-4843-4756-aff5-60c4c40df284","Type":"ContainerStarted","Data":"736ad21e3e2a866b6607566af0b930a799bbace779ed1512772c8c1b9a6f777e"}
Jul 30 13:36:16.864204 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.864034    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2" event={"ID":"c768d1f7-4843-4756-aff5-60c4c40df284","Type":"ContainerStarted","Data":"7a22893bcf74064196b79c1cb1a9730fc1f9dfd766f1abec6405b604f480bc4d"}
Jul 30 13:36:16.882167 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:16.882119    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2" podStartSLOduration=0.882105711 podStartE2EDuration="882.105711ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:36:16.879582167 +0000 UTC m=+2200.152046566" watchObservedRunningTime="2026-07-30 13:36:16.882105711 +0000 UTC m=+2200.154570108"
Jul 30 13:36:17.870219 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:17.870183    2557 generic.go:356] "Generic (PLEG): container finished" podID="c768d1f7-4843-4756-aff5-60c4c40df284" containerID="736ad21e3e2a866b6607566af0b930a799bbace779ed1512772c8c1b9a6f777e" exitCode=0
Jul 30 13:36:17.870627 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:17.870267    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2" event={"ID":"c768d1f7-4843-4756-aff5-60c4c40df284","Type":"ContainerDied","Data":"736ad21e3e2a866b6607566af0b930a799bbace779ed1512772c8c1b9a6f777e"}
Jul 30 13:36:19.002611 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:19.002576    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2"
Jul 30 13:36:19.117699 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:19.117665    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c768d1f7-4843-4756-aff5-60c4c40df284-maas-api-service-ca\") pod \"c768d1f7-4843-4756-aff5-60c4c40df284\" (UID: \"c768d1f7-4843-4756-aff5-60c4c40df284\") "
Jul 30 13:36:19.118016 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:19.117994    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c768d1f7-4843-4756-aff5-60c4c40df284-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c768d1f7-4843-4756-aff5-60c4c40df284" (UID: "c768d1f7-4843-4756-aff5-60c4c40df284"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:36:19.218997 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:19.218962    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c768d1f7-4843-4756-aff5-60c4c40df284-maas-api-service-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:36:19.880344 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:19.880315    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2"
Jul 30 13:36:19.880344 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:19.880323    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2" event={"ID":"c768d1f7-4843-4756-aff5-60c4c40df284","Type":"ContainerDied","Data":"7a22893bcf74064196b79c1cb1a9730fc1f9dfd766f1abec6405b604f480bc4d"}
Jul 30 13:36:19.880646 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:19.880362    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a22893bcf74064196b79c1cb1a9730fc1f9dfd766f1abec6405b604f480bc4d"
Jul 30 13:36:26.631646 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:26.631606    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2"]
Jul 30 13:36:26.636350 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:26.636325    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06db6c7f-rjct2"]
Jul 30 13:36:27.183236 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:27.183201    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c768d1f7-4843-4756-aff5-60c4c40df284" path="/var/lib/kubelet/pods/c768d1f7-4843-4756-aff5-60c4c40df284/volumes"
Jul 30 13:36:29.537047 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:29.537012    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:36:33.451306 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:33.451270    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"]
Jul 30 13:36:33.451823 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:33.451545    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerName="istio-proxy" containerID="cri-o://ab304667c1c8a5ca6ed0b066aea7e9c92c283f634545cabfe9d62afe9263fb8c" gracePeriod=30
Jul 30 13:36:36.332038 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:36.332003    2557 patch_prober.go:28] interesting pod/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 13:36:36.332422 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:36.332072    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 13:36:38.701022 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.701000    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:36:38.790312 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790225    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-workload-socket\") pod \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") "
Jul 30 13:36:38.790312 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790279    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-credential-socket\") pod \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") "
Jul 30 13:36:38.790542 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790336    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-workload-certs\") pod \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") "
Jul 30 13:36:38.790542 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790389    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istiod-ca-cert\") pod \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") "
Jul 30 13:36:38.790542 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790431    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-data\") pod \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") "
Jul 30 13:36:38.790542 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790453    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-token\") pod \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") "
Jul 30 13:36:38.790542 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790483    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7qvz\" (UniqueName: \"kubernetes.io/projected/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-kube-api-access-n7qvz\") pod \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") "
Jul 30 13:36:38.790542 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790516    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-podinfo\") pod \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") "
Jul 30 13:36:38.790542 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790545    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-envoy\") pod \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\" (UID: \"a1837d7d-e9b3-4b84-9986-0a1942b9d97d\") "
Jul 30 13:36:38.790949 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790656    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a1837d7d-e9b3-4b84-9986-0a1942b9d97d" (UID: "a1837d7d-e9b3-4b84-9986-0a1942b9d97d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:36:38.790949 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790699    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a1837d7d-e9b3-4b84-9986-0a1942b9d97d" (UID: "a1837d7d-e9b3-4b84-9986-0a1942b9d97d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:36:38.790949 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790732    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a1837d7d-e9b3-4b84-9986-0a1942b9d97d" (UID: "a1837d7d-e9b3-4b84-9986-0a1942b9d97d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:36:38.790949 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790945    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-credential-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:36:38.791131 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790965    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-workload-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:36:38.791131 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790979    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-workload-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:36:38.791131 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.790974    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a1837d7d-e9b3-4b84-9986-0a1942b9d97d" (UID: "a1837d7d-e9b3-4b84-9986-0a1942b9d97d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:36:38.791401 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.791380    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-data" (OuterVolumeSpecName: "istio-data") pod "a1837d7d-e9b3-4b84-9986-0a1942b9d97d" (UID: "a1837d7d-e9b3-4b84-9986-0a1942b9d97d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:36:38.793214 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.793173    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-token" (OuterVolumeSpecName: "istio-token") pod "a1837d7d-e9b3-4b84-9986-0a1942b9d97d" (UID: "a1837d7d-e9b3-4b84-9986-0a1942b9d97d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:36:38.793214 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.793184    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-kube-api-access-n7qvz" (OuterVolumeSpecName: "kube-api-access-n7qvz") pod "a1837d7d-e9b3-4b84-9986-0a1942b9d97d" (UID: "a1837d7d-e9b3-4b84-9986-0a1942b9d97d"). InnerVolumeSpecName "kube-api-access-n7qvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:36:38.793214 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.793212    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a1837d7d-e9b3-4b84-9986-0a1942b9d97d" (UID: "a1837d7d-e9b3-4b84-9986-0a1942b9d97d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:36:38.793381 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.793228    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a1837d7d-e9b3-4b84-9986-0a1942b9d97d" (UID: "a1837d7d-e9b3-4b84-9986-0a1942b9d97d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 13:36:38.892019 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.891982    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istiod-ca-cert\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:36:38.892019 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.892013    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-data\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:36:38.892019 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.892022    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-token\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:36:38.892245 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.892030    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7qvz\" (UniqueName: \"kubernetes.io/projected/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-kube-api-access-n7qvz\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:36:38.892245 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.892040    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-podinfo\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:36:38.892245 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.892048    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1837d7d-e9b3-4b84-9986-0a1942b9d97d-istio-envoy\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:36:38.968737 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.968705    2557 generic.go:356] "Generic (PLEG): container finished" podID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerID="ab304667c1c8a5ca6ed0b066aea7e9c92c283f634545cabfe9d62afe9263fb8c" exitCode=0
Jul 30 13:36:38.968897 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.968743    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" event={"ID":"a1837d7d-e9b3-4b84-9986-0a1942b9d97d","Type":"ContainerDied","Data":"ab304667c1c8a5ca6ed0b066aea7e9c92c283f634545cabfe9d62afe9263fb8c"}
Jul 30 13:36:38.968897 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.968757    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"
Jul 30 13:36:38.968897 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.968785    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5" event={"ID":"a1837d7d-e9b3-4b84-9986-0a1942b9d97d","Type":"ContainerDied","Data":"b7248cbd76ec0e8f6196391fe67c7c1dfafde3815cc0e1362306626dbad9c132"}
Jul 30 13:36:38.968897 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.968808    2557 scope.go:117] "RemoveContainer" containerID="ab304667c1c8a5ca6ed0b066aea7e9c92c283f634545cabfe9d62afe9263fb8c"
Jul 30 13:36:38.978369 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.978354    2557 scope.go:117] "RemoveContainer" containerID="b9146718f29f3c1194beb718ff51345fe6e11d1ea4a4b47cf2decdd8d884ebb0"
Jul 30 13:36:38.985923 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.985904    2557 scope.go:117] "RemoveContainer" containerID="ab304667c1c8a5ca6ed0b066aea7e9c92c283f634545cabfe9d62afe9263fb8c"
Jul 30 13:36:38.986187 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:36:38.986166    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab304667c1c8a5ca6ed0b066aea7e9c92c283f634545cabfe9d62afe9263fb8c\": container with ID starting with ab304667c1c8a5ca6ed0b066aea7e9c92c283f634545cabfe9d62afe9263fb8c not found: ID does not exist" containerID="ab304667c1c8a5ca6ed0b066aea7e9c92c283f634545cabfe9d62afe9263fb8c"
Jul 30 13:36:38.986257 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.986198    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab304667c1c8a5ca6ed0b066aea7e9c92c283f634545cabfe9d62afe9263fb8c"} err="failed to get container status \"ab304667c1c8a5ca6ed0b066aea7e9c92c283f634545cabfe9d62afe9263fb8c\": rpc error: code = NotFound desc = could not find container \"ab304667c1c8a5ca6ed0b066aea7e9c92c283f634545cabfe9d62afe9263fb8c\": container with ID starting with ab304667c1c8a5ca6ed0b066aea7e9c92c283f634545cabfe9d62afe9263fb8c not found: ID does not exist"
Jul 30 13:36:38.986257 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.986223    2557 scope.go:117] "RemoveContainer" containerID="b9146718f29f3c1194beb718ff51345fe6e11d1ea4a4b47cf2decdd8d884ebb0"
Jul 30 13:36:38.986471 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:36:38.986453    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9146718f29f3c1194beb718ff51345fe6e11d1ea4a4b47cf2decdd8d884ebb0\": container with ID starting with b9146718f29f3c1194beb718ff51345fe6e11d1ea4a4b47cf2decdd8d884ebb0 not found: ID does not exist" containerID="b9146718f29f3c1194beb718ff51345fe6e11d1ea4a4b47cf2decdd8d884ebb0"
Jul 30 13:36:38.986511 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.986477    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9146718f29f3c1194beb718ff51345fe6e11d1ea4a4b47cf2decdd8d884ebb0"} err="failed to get container status \"b9146718f29f3c1194beb718ff51345fe6e11d1ea4a4b47cf2decdd8d884ebb0\": rpc error: code = NotFound desc = could not find container \"b9146718f29f3c1194beb718ff51345fe6e11d1ea4a4b47cf2decdd8d884ebb0\": container with ID starting with b9146718f29f3c1194beb718ff51345fe6e11d1ea4a4b47cf2decdd8d884ebb0 not found: ID does not exist"
Jul 30 13:36:38.991902 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.991882    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"]
Jul 30 13:36:38.997875 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:38.997845    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-06db6c7f-openshift-default-6868497464-v6rr5"]
Jul 30 13:36:39.183269 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:36:39.183237    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" path="/var/lib/kubelet/pods/a1837d7d-e9b3-4b84-9986-0a1942b9d97d/volumes"
Jul 30 13:37:09.531824 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:37:09.531737    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:37:37.317265 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:37:37.317238    2557 scope.go:117] "RemoveContainer" containerID="40e95e23d64a102d800c8eb7eb6b4b6825bf385d8faa32afc692d8e6b04759c8"
Jul 30 13:38:37.338661 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:37.338567    2557 scope.go:117] "RemoveContainer" containerID="7b3c712e2bb95166aaa3877459b3900749fabf86e1371c616d5d05a18840ad1b"
Jul 30 13:38:50.031712 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.031672    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"]
Jul 30 13:38:50.032395 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.032375    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerName="istio-proxy"
Jul 30 13:38:50.032499 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.032398    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerName="istio-proxy"
Jul 30 13:38:50.032499 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.032432    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerName="istio-proxy"
Jul 30 13:38:50.032499 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.032441    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerName="istio-proxy"
Jul 30 13:38:50.032499 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.032481    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c768d1f7-4843-4756-aff5-60c4c40df284" containerName="revoke-keys"
Jul 30 13:38:50.032499 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.032490    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c768d1f7-4843-4756-aff5-60c4c40df284" containerName="revoke-keys"
Jul 30 13:38:50.032745 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.032642    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerName="istio-proxy"
Jul 30 13:38:50.032745 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.032656    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c768d1f7-4843-4756-aff5-60c4c40df284" containerName="revoke-keys"
Jul 30 13:38:50.036623 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.036579    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.040291 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.040248    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 13:38:50.042459 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.041977    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-85370406-openshift-default-dockercfg-dljn4\""
Jul 30 13:38:50.045348 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.045328    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"]
Jul 30 13:38:50.105816 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.105777    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzg47\" (UniqueName: \"kubernetes.io/projected/aa65f08a-ec43-4d06-b909-b96aa637289f-kube-api-access-gzg47\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.106000 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.105846    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-credential-socket\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.106000 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.105870    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-workload-certs\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.106000 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.105890    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa65f08a-ec43-4d06-b909-b96aa637289f-istiod-ca-cert\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.106000 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.105961    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-workload-socket\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.106147 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.106019    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-data\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.106147 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.106039    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-token\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.106147 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.106059    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-envoy\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.106147 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.106131    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-podinfo\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.206928 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.206894    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-podinfo\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.207121 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.206951    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzg47\" (UniqueName: \"kubernetes.io/projected/aa65f08a-ec43-4d06-b909-b96aa637289f-kube-api-access-gzg47\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.207121 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.207000    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-credential-socket\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.207121 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.207022    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-workload-certs\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.207121 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.207045    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa65f08a-ec43-4d06-b909-b96aa637289f-istiod-ca-cert\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.207350 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.207210    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-workload-socket\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.207350 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.207263    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-data\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.207350 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.207291    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-token\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.207512 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.207353    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-envoy\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.207512 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.207449    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-credential-socket\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.207512 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.207480    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-workload-certs\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.207700 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.207626    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-workload-socket\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.207882 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.207864    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-data\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.207949 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.207893    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa65f08a-ec43-4d06-b909-b96aa637289f-istiod-ca-cert\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.209459 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.209439    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-podinfo\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.209564 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.209497    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-envoy\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.216357 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.216338    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzg47\" (UniqueName: \"kubernetes.io/projected/aa65f08a-ec43-4d06-b909-b96aa637289f-kube-api-access-gzg47\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.216468 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.216450    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-token\") pod \"e2e-mt-85370406-openshift-default-5544bddc56-vfvkf\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") " pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.349220 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.349123    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:50.482924 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.482891    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"]
Jul 30 13:38:50.484497 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:38:50.484473    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa65f08a_ec43_4d06_b909_b96aa637289f.slice/crio-d3046df140ac9f28ee5ae5b92e61cb19259482c7296ff021a70478b1480d13cb WatchSource:0}: Error finding container d3046df140ac9f28ee5ae5b92e61cb19259482c7296ff021a70478b1480d13cb: Status 404 returned error can't find the container with id d3046df140ac9f28ee5ae5b92e61cb19259482c7296ff021a70478b1480d13cb
Jul 30 13:38:50.487014 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.486982    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:38:50.487110 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.487046    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:38:50.487110 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.487071    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:38:50.571839 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.571807    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf" event={"ID":"aa65f08a-ec43-4d06-b909-b96aa637289f","Type":"ContainerStarted","Data":"aec316234e8ac224e105d541bf4bd190036082ab404df6a9bd33e0b87f7f1791"}
Jul 30 13:38:50.572017 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.571859    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf" event={"ID":"aa65f08a-ec43-4d06-b909-b96aa637289f","Type":"ContainerStarted","Data":"d3046df140ac9f28ee5ae5b92e61cb19259482c7296ff021a70478b1480d13cb"}
Jul 30 13:38:50.594845 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:50.594772    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf" podStartSLOduration=0.594754392 podStartE2EDuration="594.754392ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:38:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:38:50.593485444 +0000 UTC m=+2353.865949868" watchObservedRunningTime="2026-07-30 13:38:50.594754392 +0000 UTC m=+2353.867218815"
Jul 30 13:38:51.350106 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:51.350078    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:51.355309 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:51.355287    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:51.577850 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:51.577812    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:51.578936 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:51.578911    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:38:53.239982 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:38:53.239954    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:39:06.229824 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:06.229789    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:39:10.668154 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:10.668119    2557 generic.go:356] "Generic (PLEG): container finished" podID="aa65f08a-ec43-4d06-b909-b96aa637289f" containerID="aec316234e8ac224e105d541bf4bd190036082ab404df6a9bd33e0b87f7f1791" exitCode=0
Jul 30 13:39:10.668715 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:10.668163    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf" event={"ID":"aa65f08a-ec43-4d06-b909-b96aa637289f","Type":"ContainerDied","Data":"aec316234e8ac224e105d541bf4bd190036082ab404df6a9bd33e0b87f7f1791"}
Jul 30 13:39:10.668715 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:10.668676    2557 scope.go:117] "RemoveContainer" containerID="aec316234e8ac224e105d541bf4bd190036082ab404df6a9bd33e0b87f7f1791"
Jul 30 13:39:10.669764 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:10.669732    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:39:10.669866 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:10.669802    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:39:10.669866 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:10.669831    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:39:11.678069 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:11.677357    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf" event={"ID":"aa65f08a-ec43-4d06-b909-b96aa637289f","Type":"ContainerStarted","Data":"93225db967def6b815a9c7c65b3ec1b8b206d57ac1740fa918f4011d049dc213"}
Jul 30 13:39:12.349416 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:12.349380    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:39:12.349684 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:12.349663    2557 patch_prober.go:28] interesting pod/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 30 13:39:12.349776 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:12.349718    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf" podUID="aa65f08a-ec43-4d06-b909-b96aa637289f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 30 13:39:13.349530 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:13.349494    2557 patch_prober.go:28] interesting pod/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 30 13:39:13.349975 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:13.349547    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf" podUID="aa65f08a-ec43-4d06-b909-b96aa637289f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 30 13:39:14.349816 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:14.349777    2557 patch_prober.go:28] interesting pod/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 30 13:39:14.350289 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:14.349841    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf" podUID="aa65f08a-ec43-4d06-b909-b96aa637289f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 30 13:39:15.350281 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:15.350241    2557 patch_prober.go:28] interesting pod/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 30 13:39:15.350708 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:15.350306    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf" podUID="aa65f08a-ec43-4d06-b909-b96aa637289f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 30 13:39:16.353633 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:16.353582    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:39:16.354084 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:16.353883    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:39:16.354494 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:16.354477    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:39:28.234256 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.234225    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"]
Jul 30 13:39:28.234659 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.234644    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1837d7d-e9b3-4b84-9986-0a1942b9d97d" containerName="istio-proxy"
Jul 30 13:39:28.237816 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.237796    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.241005 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.240985    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-85370406\"/\"e2e-lifecycle-model-85370406-kserve-self-signed-certs\""
Jul 30 13:39:28.242256 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.242233    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-85370406\"/\"kube-root-ca.crt\""
Jul 30 13:39:28.242364 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.242282    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-85370406\"/\"openshift-service-ca.crt\""
Jul 30 13:39:28.242364 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.242284    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-85370406\"/\"default-dockercfg-wkfjd\""
Jul 30 13:39:28.249957 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.249939    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"]
Jul 30 13:39:28.335168 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.335134    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-dshm\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.335168 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.335169    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-model-cache\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.335422 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.335215    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-tls-certs\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.335422 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.335277    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-tmp-dir\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.335422 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.335399    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-home\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.335637 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.335437    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5phss\" (UniqueName: \"kubernetes.io/projected/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-kube-api-access-5phss\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.436338 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.436304    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-tls-certs\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.436338 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.436344    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-tmp-dir\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.436620 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.436392    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-home\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.436620 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.436412    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5phss\" (UniqueName: \"kubernetes.io/projected/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-kube-api-access-5phss\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.436620 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.436430    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-dshm\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.436620 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.436444    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-model-cache\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.436908 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.436885    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-home\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.436980 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.436941    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-model-cache\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.436980 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.436971    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-tmp-dir\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.438756 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.438740    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-dshm\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.438923 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.438906    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-tls-certs\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.444308 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.444284    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5phss\" (UniqueName: \"kubernetes.io/projected/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-kube-api-access-5phss\") pod \"e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") " pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.547753 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.547669    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:28.679815 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.679789    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"]
Jul 30 13:39:28.681452 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:39:28.681423    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff055812_e9ae_41ea_9e10_a0e6ccf3cbb9.slice/crio-49eb0efd8453a54339c7410c797bbf71e7965192edd06f5de1c78c211ccd21a3 WatchSource:0}: Error finding container 49eb0efd8453a54339c7410c797bbf71e7965192edd06f5de1c78c211ccd21a3: Status 404 returned error can't find the container with id 49eb0efd8453a54339c7410c797bbf71e7965192edd06f5de1c78c211ccd21a3
Jul 30 13:39:28.759443 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.759408    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4" event={"ID":"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9","Type":"ContainerStarted","Data":"f097060c1ce2d2c809c18d5e17602fe1bc66494476385f8bd81cf3c0bfb5fbf4"}
Jul 30 13:39:28.759583 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.759449    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4" event={"ID":"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9","Type":"ContainerStarted","Data":"49eb0efd8453a54339c7410c797bbf71e7965192edd06f5de1c78c211ccd21a3"}
Jul 30 13:39:28.776476 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:28.776431    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4" podStartSLOduration=0.776414959 podStartE2EDuration="776.414959ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:39:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:39:28.775929252 +0000 UTC m=+2392.048393651" watchObservedRunningTime="2026-07-30 13:39:28.776414959 +0000 UTC m=+2392.048879357"
Jul 30 13:39:31.639131 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:31.639092    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:39:37.356725 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:37.356691    2557 scope.go:117] "RemoveContainer" containerID="e5898e444f378ec6886dbcfd04735631d7fd80df0157ffdd339cda8fe0948cfb"
Jul 30 13:39:37.364914 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:37.364896    2557 scope.go:117] "RemoveContainer" containerID="1e67f96e218ac9d483e1216bad9a7f705378d1f51b30aabcf5faa0c86afac8ec"
Jul 30 13:39:38.548701 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:38.548666    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:38.549150 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:38.548949    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:38.561639 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:38.561615    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:38.818896 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:38.818813    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:39:58.142407 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:39:58.142365    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:40:07.697816 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:07.697731    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788"]
Jul 30 13:40:07.701411 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:07.701392    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788"
Jul 30 13:40:07.704355 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:07.704331    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxdpz\""
Jul 30 13:40:07.704472 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:07.704376    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 13:40:07.708835 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:07.708806    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788"]
Jul 30 13:40:07.788791 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:07.788755    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e25ed8a-6b52-4e25-beca-327dca3759b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-85370406-9l788\" (UID: \"7e25ed8a-6b52-4e25-beca-327dca3759b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788"
Jul 30 13:40:07.889958 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:07.889917    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e25ed8a-6b52-4e25-beca-327dca3759b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-85370406-9l788\" (UID: \"7e25ed8a-6b52-4e25-beca-327dca3759b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788"
Jul 30 13:40:07.890655 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:07.890592    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e25ed8a-6b52-4e25-beca-327dca3759b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-85370406-9l788\" (UID: \"7e25ed8a-6b52-4e25-beca-327dca3759b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788"
Jul 30 13:40:08.013130 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:08.013038    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788"
Jul 30 13:40:08.152425 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:08.152398    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788"]
Jul 30 13:40:08.154815 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:40:08.154786    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e25ed8a_6b52_4e25_beca_327dca3759b4.slice/crio-80f2ba1ccabba17ca2282b45f66b5ca3e8499048c4c04543c01c8c9dd1ff43df WatchSource:0}: Error finding container 80f2ba1ccabba17ca2282b45f66b5ca3e8499048c4c04543c01c8c9dd1ff43df: Status 404 returned error can't find the container with id 80f2ba1ccabba17ca2282b45f66b5ca3e8499048c4c04543c01c8c9dd1ff43df
Jul 30 13:40:08.951875 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:08.951842    2557 generic.go:356] "Generic (PLEG): container finished" podID="7e25ed8a-6b52-4e25-beca-327dca3759b4" containerID="d91fd5775f660aa1193f19a783ead8cae857a95979564c127ce55805a8544eaa" exitCode=0
Jul 30 13:40:08.952285 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:08.951887    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788" event={"ID":"7e25ed8a-6b52-4e25-beca-327dca3759b4","Type":"ContainerDied","Data":"d91fd5775f660aa1193f19a783ead8cae857a95979564c127ce55805a8544eaa"}
Jul 30 13:40:08.952285 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:08.951936    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788" event={"ID":"7e25ed8a-6b52-4e25-beca-327dca3759b4","Type":"ContainerStarted","Data":"80f2ba1ccabba17ca2282b45f66b5ca3e8499048c4c04543c01c8c9dd1ff43df"}
Jul 30 13:40:10.091799 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:10.091772    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788"
Jul 30 13:40:10.211583 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:10.211496    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e25ed8a-6b52-4e25-beca-327dca3759b4-maas-api-service-ca\") pod \"7e25ed8a-6b52-4e25-beca-327dca3759b4\" (UID: \"7e25ed8a-6b52-4e25-beca-327dca3759b4\") "
Jul 30 13:40:10.211896 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:10.211872    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e25ed8a-6b52-4e25-beca-327dca3759b4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7e25ed8a-6b52-4e25-beca-327dca3759b4" (UID: "7e25ed8a-6b52-4e25-beca-327dca3759b4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:40:10.313185 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:10.313147    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e25ed8a-6b52-4e25-beca-327dca3759b4-maas-api-service-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:40:10.962118 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:10.962086    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788" event={"ID":"7e25ed8a-6b52-4e25-beca-327dca3759b4","Type":"ContainerDied","Data":"80f2ba1ccabba17ca2282b45f66b5ca3e8499048c4c04543c01c8c9dd1ff43df"}
Jul 30 13:40:10.962118 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:10.962122    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788"
Jul 30 13:40:10.962327 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:10.962135    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80f2ba1ccabba17ca2282b45f66b5ca3e8499048c4c04543c01c8c9dd1ff43df"
Jul 30 13:40:16.630201 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:16.630165    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:40:18.423065 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:18.423017    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788"]
Jul 30 13:40:18.426406 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:18.426375    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85370406-9l788"]
Jul 30 13:40:19.182801 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:19.182769    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e25ed8a-6b52-4e25-beca-327dca3759b4" path="/var/lib/kubelet/pods/7e25ed8a-6b52-4e25-beca-327dca3759b4/volumes"
Jul 30 13:40:25.243035 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:25.242998    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:40:26.720536 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:26.720502    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"]
Jul 30 13:40:41.739734 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:41.739673    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4" podUID="ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9" containerName="main" containerID="cri-o://f097060c1ce2d2c809c18d5e17602fe1bc66494476385f8bd81cf3c0bfb5fbf4" gracePeriod=45
Jul 30 13:40:42.001002 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.000933    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:40:42.094190 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.094157    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-home\") pod \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") "
Jul 30 13:40:42.094190 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.094194    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5phss\" (UniqueName: \"kubernetes.io/projected/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-kube-api-access-5phss\") pod \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") "
Jul 30 13:40:42.094431 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.094235    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-model-cache\") pod \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") "
Jul 30 13:40:42.094431 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.094260    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-tls-certs\") pod \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") "
Jul 30 13:40:42.094431 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.094275    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-tmp-dir\") pod \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") "
Jul 30 13:40:42.094431 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.094321    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-dshm\") pod \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\" (UID: \"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9\") "
Jul 30 13:40:42.094678 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.094455    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-home" (OuterVolumeSpecName: "home") pod "ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9" (UID: "ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:40:42.094678 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.094532    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9" (UID: "ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:40:42.094787 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.094687    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-home\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:40:42.094787 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.094708    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-model-cache\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:40:42.094908 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.094880    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9" (UID: "ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:40:42.096562 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.096534    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-kube-api-access-5phss" (OuterVolumeSpecName: "kube-api-access-5phss") pod "ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9" (UID: "ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9"). InnerVolumeSpecName "kube-api-access-5phss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:40:42.096900 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.096874    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9" (UID: "ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 13:40:42.096984 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.096915    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-dshm" (OuterVolumeSpecName: "dshm") pod "ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9" (UID: "ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:40:42.110351 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.110323    2557 generic.go:356] "Generic (PLEG): container finished" podID="ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9" containerID="f097060c1ce2d2c809c18d5e17602fe1bc66494476385f8bd81cf3c0bfb5fbf4" exitCode=0
Jul 30 13:40:42.110459 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.110367    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"
Jul 30 13:40:42.110459 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.110403    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4" event={"ID":"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9","Type":"ContainerDied","Data":"f097060c1ce2d2c809c18d5e17602fe1bc66494476385f8bd81cf3c0bfb5fbf4"}
Jul 30 13:40:42.110459 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.110451    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4" event={"ID":"ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9","Type":"ContainerDied","Data":"49eb0efd8453a54339c7410c797bbf71e7965192edd06f5de1c78c211ccd21a3"}
Jul 30 13:40:42.110566 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.110475    2557 scope.go:117] "RemoveContainer" containerID="f097060c1ce2d2c809c18d5e17602fe1bc66494476385f8bd81cf3c0bfb5fbf4"
Jul 30 13:40:42.118949 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.118932    2557 scope.go:117] "RemoveContainer" containerID="f097060c1ce2d2c809c18d5e17602fe1bc66494476385f8bd81cf3c0bfb5fbf4"
Jul 30 13:40:42.119204 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:40:42.119186    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f097060c1ce2d2c809c18d5e17602fe1bc66494476385f8bd81cf3c0bfb5fbf4\": container with ID starting with f097060c1ce2d2c809c18d5e17602fe1bc66494476385f8bd81cf3c0bfb5fbf4 not found: ID does not exist" containerID="f097060c1ce2d2c809c18d5e17602fe1bc66494476385f8bd81cf3c0bfb5fbf4"
Jul 30 13:40:42.119262 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.119214    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f097060c1ce2d2c809c18d5e17602fe1bc66494476385f8bd81cf3c0bfb5fbf4"} err="failed to get container status \"f097060c1ce2d2c809c18d5e17602fe1bc66494476385f8bd81cf3c0bfb5fbf4\": rpc error: code = NotFound desc = could not find container \"f097060c1ce2d2c809c18d5e17602fe1bc66494476385f8bd81cf3c0bfb5fbf4\": container with ID starting with f097060c1ce2d2c809c18d5e17602fe1bc66494476385f8bd81cf3c0bfb5fbf4 not found: ID does not exist"
Jul 30 13:40:42.132034 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.132007    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"]
Jul 30 13:40:42.133501 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.133483    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-85370406/e2e-lifecycle-model-85370406-kserve-7df658b679-sdcg4"]
Jul 30 13:40:42.195390 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.195350    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5phss\" (UniqueName: \"kubernetes.io/projected/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-kube-api-access-5phss\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:40:42.195390 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.195380    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-tls-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:40:42.195688 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.195410    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-tmp-dir\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:40:42.195688 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:42.195421    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9-dshm\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:40:43.184046 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:40:43.184013    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9" path="/var/lib/kubelet/pods/ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9/volumes"
Jul 30 13:41:10.007071 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:10.007038    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"]
Jul 30 13:41:10.007575 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:10.007259    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf" podUID="aa65f08a-ec43-4d06-b909-b96aa637289f" containerName="istio-proxy" containerID="cri-o://93225db967def6b815a9c7c65b3ec1b8b206d57ac1740fa918f4011d049dc213" gracePeriod=30
Jul 30 13:41:12.933802 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:12.933764    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:41:15.259687 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.259663    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:41:15.266681 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.266652    2557 generic.go:356] "Generic (PLEG): container finished" podID="aa65f08a-ec43-4d06-b909-b96aa637289f" containerID="93225db967def6b815a9c7c65b3ec1b8b206d57ac1740fa918f4011d049dc213" exitCode=0
Jul 30 13:41:15.266811 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.266697    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"
Jul 30 13:41:15.266811 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.266729    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf" event={"ID":"aa65f08a-ec43-4d06-b909-b96aa637289f","Type":"ContainerDied","Data":"93225db967def6b815a9c7c65b3ec1b8b206d57ac1740fa918f4011d049dc213"}
Jul 30 13:41:15.266811 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.266780    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf" event={"ID":"aa65f08a-ec43-4d06-b909-b96aa637289f","Type":"ContainerDied","Data":"d3046df140ac9f28ee5ae5b92e61cb19259482c7296ff021a70478b1480d13cb"}
Jul 30 13:41:15.266811 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.266803    2557 scope.go:117] "RemoveContainer" containerID="93225db967def6b815a9c7c65b3ec1b8b206d57ac1740fa918f4011d049dc213"
Jul 30 13:41:15.274891 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.274874    2557 scope.go:117] "RemoveContainer" containerID="aec316234e8ac224e105d541bf4bd190036082ab404df6a9bd33e0b87f7f1791"
Jul 30 13:41:15.283548 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.283524    2557 scope.go:117] "RemoveContainer" containerID="93225db967def6b815a9c7c65b3ec1b8b206d57ac1740fa918f4011d049dc213"
Jul 30 13:41:15.284050 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:41:15.283904    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93225db967def6b815a9c7c65b3ec1b8b206d57ac1740fa918f4011d049dc213\": container with ID starting with 93225db967def6b815a9c7c65b3ec1b8b206d57ac1740fa918f4011d049dc213 not found: ID does not exist" containerID="93225db967def6b815a9c7c65b3ec1b8b206d57ac1740fa918f4011d049dc213"
Jul 30 13:41:15.284050 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.283954    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93225db967def6b815a9c7c65b3ec1b8b206d57ac1740fa918f4011d049dc213"} err="failed to get container status \"93225db967def6b815a9c7c65b3ec1b8b206d57ac1740fa918f4011d049dc213\": rpc error: code = NotFound desc = could not find container \"93225db967def6b815a9c7c65b3ec1b8b206d57ac1740fa918f4011d049dc213\": container with ID starting with 93225db967def6b815a9c7c65b3ec1b8b206d57ac1740fa918f4011d049dc213 not found: ID does not exist"
Jul 30 13:41:15.284050 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.283978    2557 scope.go:117] "RemoveContainer" containerID="aec316234e8ac224e105d541bf4bd190036082ab404df6a9bd33e0b87f7f1791"
Jul 30 13:41:15.284273 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:41:15.284253    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aec316234e8ac224e105d541bf4bd190036082ab404df6a9bd33e0b87f7f1791\": container with ID starting with aec316234e8ac224e105d541bf4bd190036082ab404df6a9bd33e0b87f7f1791 not found: ID does not exist" containerID="aec316234e8ac224e105d541bf4bd190036082ab404df6a9bd33e0b87f7f1791"
Jul 30 13:41:15.284332 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.284280    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aec316234e8ac224e105d541bf4bd190036082ab404df6a9bd33e0b87f7f1791"} err="failed to get container status \"aec316234e8ac224e105d541bf4bd190036082ab404df6a9bd33e0b87f7f1791\": rpc error: code = NotFound desc = could not find container \"aec316234e8ac224e105d541bf4bd190036082ab404df6a9bd33e0b87f7f1791\": container with ID starting with aec316234e8ac224e105d541bf4bd190036082ab404df6a9bd33e0b87f7f1791 not found: ID does not exist"
Jul 30 13:41:15.394781 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.394751    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-envoy\") pod \"aa65f08a-ec43-4d06-b909-b96aa637289f\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") "
Jul 30 13:41:15.394969 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.394793    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-credential-socket\") pod \"aa65f08a-ec43-4d06-b909-b96aa637289f\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") "
Jul 30 13:41:15.394969 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.394818    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-data\") pod \"aa65f08a-ec43-4d06-b909-b96aa637289f\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") "
Jul 30 13:41:15.394969 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.394859    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-podinfo\") pod \"aa65f08a-ec43-4d06-b909-b96aa637289f\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") "
Jul 30 13:41:15.394969 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.394897    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-token\") pod \"aa65f08a-ec43-4d06-b909-b96aa637289f\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") "
Jul 30 13:41:15.395200 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.395122    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aa65f08a-ec43-4d06-b909-b96aa637289f" (UID: "aa65f08a-ec43-4d06-b909-b96aa637289f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:41:15.395296 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.395259    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzg47\" (UniqueName: \"kubernetes.io/projected/aa65f08a-ec43-4d06-b909-b96aa637289f-kube-api-access-gzg47\") pod \"aa65f08a-ec43-4d06-b909-b96aa637289f\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") "
Jul 30 13:41:15.395378 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.395321    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-workload-certs\") pod \"aa65f08a-ec43-4d06-b909-b96aa637289f\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") "
Jul 30 13:41:15.395378 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.395352    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa65f08a-ec43-4d06-b909-b96aa637289f-istiod-ca-cert\") pod \"aa65f08a-ec43-4d06-b909-b96aa637289f\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") "
Jul 30 13:41:15.395493 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.395408    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-workload-socket\") pod \"aa65f08a-ec43-4d06-b909-b96aa637289f\" (UID: \"aa65f08a-ec43-4d06-b909-b96aa637289f\") "
Jul 30 13:41:15.395952 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.395762    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aa65f08a-ec43-4d06-b909-b96aa637289f" (UID: "aa65f08a-ec43-4d06-b909-b96aa637289f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:41:15.396064 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.395999    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-data" (OuterVolumeSpecName: "istio-data") pod "aa65f08a-ec43-4d06-b909-b96aa637289f" (UID: "aa65f08a-ec43-4d06-b909-b96aa637289f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:41:15.396277 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.396249    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa65f08a-ec43-4d06-b909-b96aa637289f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aa65f08a-ec43-4d06-b909-b96aa637289f" (UID: "aa65f08a-ec43-4d06-b909-b96aa637289f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:41:15.396277 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.396265    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-workload-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:41:15.396460 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.396290    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-credential-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:41:15.396460 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.396305    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-data\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:41:15.396460 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.396354    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aa65f08a-ec43-4d06-b909-b96aa637289f" (UID: "aa65f08a-ec43-4d06-b909-b96aa637289f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:41:15.397292 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.397263    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aa65f08a-ec43-4d06-b909-b96aa637289f" (UID: "aa65f08a-ec43-4d06-b909-b96aa637289f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 13:41:15.397396 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.397269    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aa65f08a-ec43-4d06-b909-b96aa637289f" (UID: "aa65f08a-ec43-4d06-b909-b96aa637289f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:41:15.397456 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.397414    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-token" (OuterVolumeSpecName: "istio-token") pod "aa65f08a-ec43-4d06-b909-b96aa637289f" (UID: "aa65f08a-ec43-4d06-b909-b96aa637289f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:41:15.398177 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.398149    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa65f08a-ec43-4d06-b909-b96aa637289f-kube-api-access-gzg47" (OuterVolumeSpecName: "kube-api-access-gzg47") pod "aa65f08a-ec43-4d06-b909-b96aa637289f" (UID: "aa65f08a-ec43-4d06-b909-b96aa637289f"). InnerVolumeSpecName "kube-api-access-gzg47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:41:15.496999 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.496948    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-token\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:41:15.496999 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.496992    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzg47\" (UniqueName: \"kubernetes.io/projected/aa65f08a-ec43-4d06-b909-b96aa637289f-kube-api-access-gzg47\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:41:15.496999 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.497003    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa65f08a-ec43-4d06-b909-b96aa637289f-istiod-ca-cert\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:41:15.497248 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.497014    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-workload-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:41:15.497248 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.497022    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-envoy\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:41:15.497248 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.497031    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa65f08a-ec43-4d06-b909-b96aa637289f-istio-podinfo\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:41:15.590385 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.590341    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"]
Jul 30 13:41:15.595809 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:15.595778    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-85370406-openshift-default-5544bddc56-vfvkf"]
Jul 30 13:41:17.184106 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:17.184067    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa65f08a-ec43-4d06-b909-b96aa637289f" path="/var/lib/kubelet/pods/aa65f08a-ec43-4d06-b909-b96aa637289f/volumes"
Jul 30 13:41:36.645775 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:36.645682    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:41:46.136579 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:41:46.136545    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:42:25.600229 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.600196    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"]
Jul 30 13:42:25.600760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.600530    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa65f08a-ec43-4d06-b909-b96aa637289f" containerName="istio-proxy"
Jul 30 13:42:25.600760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.600542    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa65f08a-ec43-4d06-b909-b96aa637289f" containerName="istio-proxy"
Jul 30 13:42:25.600760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.600559    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9" containerName="main"
Jul 30 13:42:25.600760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.600566    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9" containerName="main"
Jul 30 13:42:25.600760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.600577    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e25ed8a-6b52-4e25-beca-327dca3759b4" containerName="revoke-keys"
Jul 30 13:42:25.600760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.600582    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e25ed8a-6b52-4e25-beca-327dca3759b4" containerName="revoke-keys"
Jul 30 13:42:25.600760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.600589    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa65f08a-ec43-4d06-b909-b96aa637289f" containerName="istio-proxy"
Jul 30 13:42:25.600760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.600605    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa65f08a-ec43-4d06-b909-b96aa637289f" containerName="istio-proxy"
Jul 30 13:42:25.600760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.600676    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa65f08a-ec43-4d06-b909-b96aa637289f" containerName="istio-proxy"
Jul 30 13:42:25.600760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.600686    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e25ed8a-6b52-4e25-beca-327dca3759b4" containerName="revoke-keys"
Jul 30 13:42:25.600760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.600693    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa65f08a-ec43-4d06-b909-b96aa637289f" containerName="istio-proxy"
Jul 30 13:42:25.600760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.600700    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff055812-e9ae-41ea-9e10-a0e6ccf3cbb9" containerName="main"
Jul 30 13:42:25.603398 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.603380    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.606746 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.606720    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 13:42:25.606857 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.606772    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-628283-openshift-default-dockercfg-zmtrc\""
Jul 30 13:42:25.626283 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.626256    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"]
Jul 30 13:42:25.786498 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.786458    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-workload-certs\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.786718 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.786504    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-credential-socket\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.786718 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.786559    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbq58\" (UniqueName: \"kubernetes.io/projected/d152f512-99ac-4365-b565-b775b22a374d-kube-api-access-sbq58\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.786718 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.786580    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-workload-socket\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.786718 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.786624    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d152f512-99ac-4365-b565-b775b22a374d-istio-token\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.786718 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.786655    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d152f512-99ac-4365-b565-b775b22a374d-istiod-ca-cert\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.786910 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.786737    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d152f512-99ac-4365-b565-b775b22a374d-istio-podinfo\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.786910 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.786774    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-istio-data\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.786910 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.786797    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-istio-envoy\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.887454 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.887367    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-credential-socket\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.887454 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.887415    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbq58\" (UniqueName: \"kubernetes.io/projected/d152f512-99ac-4365-b565-b775b22a374d-kube-api-access-sbq58\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.887454 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.887453    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-workload-socket\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.887787 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.887471    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d152f512-99ac-4365-b565-b775b22a374d-istio-token\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.887787 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.887502    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d152f512-99ac-4365-b565-b775b22a374d-istiod-ca-cert\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.887787 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.887672    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d152f512-99ac-4365-b565-b775b22a374d-istio-podinfo\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.887787 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.887721    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-istio-data\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.887787 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.887764    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-istio-envoy\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.888020 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.887823    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-workload-certs\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.888020 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.887839    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-credential-socket\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.888020 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.887932    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-workload-socket\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.888020 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.888002    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-istio-data\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.888155 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.888129    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-workload-certs\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.888282 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.888253    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d152f512-99ac-4365-b565-b775b22a374d-istiod-ca-cert\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.890048 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.890019    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-istio-envoy\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.890258 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.890242    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d152f512-99ac-4365-b565-b775b22a374d-istio-podinfo\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.897875 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.897855    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d152f512-99ac-4365-b565-b775b22a374d-istio-token\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.898096 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.898079    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbq58\" (UniqueName: \"kubernetes.io/projected/d152f512-99ac-4365-b565-b775b22a374d-kube-api-access-sbq58\") pod \"e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") " pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:25.913941 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:25.913917    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:26.052075 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:26.052043    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"]
Jul 30 13:42:26.055363 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:42:26.055335    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd152f512_99ac_4365_b565_b775b22a374d.slice/crio-6b1c1ea4ccb35be513c5b2829eb5615ddc1093d4fb57ca9ddf0cfcc6a5ae89a1 WatchSource:0}: Error finding container 6b1c1ea4ccb35be513c5b2829eb5615ddc1093d4fb57ca9ddf0cfcc6a5ae89a1: Status 404 returned error can't find the container with id 6b1c1ea4ccb35be513c5b2829eb5615ddc1093d4fb57ca9ddf0cfcc6a5ae89a1
Jul 30 13:42:26.057745 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:26.057707    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:42:26.057830 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:26.057795    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:42:26.057868 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:26.057836    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:42:26.598705 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:26.598674    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj" event={"ID":"d152f512-99ac-4365-b565-b775b22a374d","Type":"ContainerStarted","Data":"cabb5bd281da027a046361fb51418805ed0a6f255fee1e34a879d19f52c756bf"}
Jul 30 13:42:26.598880 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:26.598720    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj" event={"ID":"d152f512-99ac-4365-b565-b775b22a374d","Type":"ContainerStarted","Data":"6b1c1ea4ccb35be513c5b2829eb5615ddc1093d4fb57ca9ddf0cfcc6a5ae89a1"}
Jul 30 13:42:26.621901 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:26.621849    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj" podStartSLOduration=1.6218356269999998 podStartE2EDuration="1.621835627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:42:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:42:26.621048605 +0000 UTC m=+2569.893513016" watchObservedRunningTime="2026-07-30 13:42:26.621835627 +0000 UTC m=+2569.894300015"
Jul 30 13:42:26.914376 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:26.914346    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:27.919947 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:27.919916    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:28.607611 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:28.607570    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:28.608628 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:28.608579    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:35.541079 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:35.541048    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:42:37.410265 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:37.410236    2557 scope.go:117] "RemoveContainer" containerID="736ad21e3e2a866b6607566af0b930a799bbace779ed1512772c8c1b9a6f777e"
Jul 30 13:42:43.608841 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:43.608805    2557 patch_prober.go:28] interesting pod/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 30 13:42:43.609281 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:43.608867    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj" podUID="d152f512-99ac-4365-b565-b775b22a374d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 30 13:42:43.678092 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:43.678062    2557 generic.go:356] "Generic (PLEG): container finished" podID="d152f512-99ac-4365-b565-b775b22a374d" containerID="cabb5bd281da027a046361fb51418805ed0a6f255fee1e34a879d19f52c756bf" exitCode=0
Jul 30 13:42:43.678262 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:43.678135    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj" event={"ID":"d152f512-99ac-4365-b565-b775b22a374d","Type":"ContainerDied","Data":"cabb5bd281da027a046361fb51418805ed0a6f255fee1e34a879d19f52c756bf"}
Jul 30 13:42:43.678549 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:43.678536    2557 scope.go:117] "RemoveContainer" containerID="cabb5bd281da027a046361fb51418805ed0a6f255fee1e34a879d19f52c756bf"
Jul 30 13:42:43.679497 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:43.679473    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:42:43.679546 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:43.679537    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:42:43.679581 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:43.679567    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:42:44.687646 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:44.687583    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj" event={"ID":"d152f512-99ac-4365-b565-b775b22a374d","Type":"ContainerStarted","Data":"755a8e05901d4a0f754edd6bc5bb39af24a1802105d64a95a51ec17a5bf39a3c"}
Jul 30 13:42:44.914454 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:44.914414    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:44.914664 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:44.914631    2557 patch_prober.go:28] interesting pod/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 30 13:42:44.914717 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:44.914679    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj" podUID="d152f512-99ac-4365-b565-b775b22a374d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 30 13:42:45.914624 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:45.914560    2557 patch_prober.go:28] interesting pod/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 30 13:42:45.915017 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:45.914647    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj" podUID="d152f512-99ac-4365-b565-b775b22a374d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 30 13:42:46.915319 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:46.915284    2557 patch_prober.go:28] interesting pod/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 30 13:42:46.915730 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:46.915336    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj" podUID="d152f512-99ac-4365-b565-b775b22a374d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 30 13:42:47.915187 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:47.915148    2557 patch_prober.go:28] interesting pod/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 30 13:42:47.915367 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:47.915208    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj" podUID="d152f512-99ac-4365-b565-b775b22a374d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 30 13:42:48.919360 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:48.919327    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:48.919864 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:48.919690    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:48.920318 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:48.920299    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:42:54.217959 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.217919    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"]
Jul 30 13:42:54.221587 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.221566    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.224750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.224732    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-bdee41-openshift-default-dockercfg-pfd8b\""
Jul 30 13:42:54.233808 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.233780    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"]
Jul 30 13:42:54.346452 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.346412    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-podinfo\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.346452 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.346454    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istiod-ca-cert\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.346689 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.346479    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-credential-socket\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.346689 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.346557    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-workload-certs\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.346689 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.346585    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlfvd\" (UniqueName: \"kubernetes.io/projected/ba5192ec-9b5e-4603-a72d-d6206a2a1823-kube-api-access-wlfvd\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.346689 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.346632    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-data\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.346689 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.346670    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-workload-socket\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.346858 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.346708    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-envoy\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.346858 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.346742    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-token\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.447898 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.447864    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-podinfo\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.448098 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.447911    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istiod-ca-cert\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.448098 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.447934    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-credential-socket\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.448098 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.448057    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-workload-certs\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.448275 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.448098    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlfvd\" (UniqueName: \"kubernetes.io/projected/ba5192ec-9b5e-4603-a72d-d6206a2a1823-kube-api-access-wlfvd\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.448275 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.448134    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-data\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.448275 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.448170    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-workload-socket\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.448275 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.448213    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-envoy\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.448275 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.448271    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-token\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.448521 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.448472    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-credential-socket\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.448521 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.448500    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-workload-certs\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.448666 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.448643    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-workload-socket\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.448725 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.448703    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istiod-ca-cert\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.448775 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.448748    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-data\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.450424 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.450400    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-envoy\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.450541 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.450526    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-podinfo\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.457258 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.457237    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlfvd\" (UniqueName: \"kubernetes.io/projected/ba5192ec-9b5e-4603-a72d-d6206a2a1823-kube-api-access-wlfvd\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.457392 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.457373    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-token\") pod \"e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") " pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.531921 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.531833    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:54.670508 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.670483    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"]
Jul 30 13:42:54.673061 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:42:54.673036    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba5192ec_9b5e_4603_a72d_d6206a2a1823.slice/crio-971323c930fe67a1e504907d16163b984a7edab04ac61f81d83a4dca99950416 WatchSource:0}: Error finding container 971323c930fe67a1e504907d16163b984a7edab04ac61f81d83a4dca99950416: Status 404 returned error can't find the container with id 971323c930fe67a1e504907d16163b984a7edab04ac61f81d83a4dca99950416
Jul 30 13:42:54.675060 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.675031    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:42:54.675161 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.675110    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:42:54.675161 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.675137    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:42:54.734648 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:54.734621    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x" event={"ID":"ba5192ec-9b5e-4603-a72d-d6206a2a1823","Type":"ContainerStarted","Data":"971323c930fe67a1e504907d16163b984a7edab04ac61f81d83a4dca99950416"}
Jul 30 13:42:55.741128 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:55.741090    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x" event={"ID":"ba5192ec-9b5e-4603-a72d-d6206a2a1823","Type":"ContainerStarted","Data":"f196e0fd62625ef17e99161ada6eaaf2c37c7ef64344706a4172824378fb322a"}
Jul 30 13:42:55.763392 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:55.763332    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x" podStartSLOduration=1.763312236 podStartE2EDuration="1.763312236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:42:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:42:55.761546347 +0000 UTC m=+2599.034010766" watchObservedRunningTime="2026-07-30 13:42:55.763312236 +0000 UTC m=+2599.035776633"
Jul 30 13:42:56.532411 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:56.532377    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:56.537504 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:56.537481    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:56.745982 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:56.745946    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:56.746905 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:56.746886    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:42:58.752616 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.752571    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"]
Jul 30 13:42:58.756380 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.756355    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:42:58.759166 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.759129    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-bdee41-serving-cert\""
Jul 30 13:42:58.759279 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.759215    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 13:42:58.759346 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.759295    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dnpqq\""
Jul 30 13:42:58.760485 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.760468    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 13:42:58.768434 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.768412    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"]
Jul 30 13:42:58.886157 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.886072    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/536d03cc-cbf1-4493-aa35-0721241e3f28-openshift-service-ca\") pod \"maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n\" (UID: \"536d03cc-cbf1-4493-aa35-0721241e3f28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:42:58.886157 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.886112    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss5jt\" (UniqueName: \"kubernetes.io/projected/536d03cc-cbf1-4493-aa35-0721241e3f28-kube-api-access-ss5jt\") pod \"maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n\" (UID: \"536d03cc-cbf1-4493-aa35-0721241e3f28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:42:58.886401 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.886235    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/536d03cc-cbf1-4493-aa35-0721241e3f28-maas-api-tls\") pod \"maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n\" (UID: \"536d03cc-cbf1-4493-aa35-0721241e3f28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:42:58.987760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.987728    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/536d03cc-cbf1-4493-aa35-0721241e3f28-openshift-service-ca\") pod \"maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n\" (UID: \"536d03cc-cbf1-4493-aa35-0721241e3f28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:42:58.987760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.987762    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss5jt\" (UniqueName: \"kubernetes.io/projected/536d03cc-cbf1-4493-aa35-0721241e3f28-kube-api-access-ss5jt\") pod \"maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n\" (UID: \"536d03cc-cbf1-4493-aa35-0721241e3f28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:42:58.987962 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.987797    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/536d03cc-cbf1-4493-aa35-0721241e3f28-maas-api-tls\") pod \"maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n\" (UID: \"536d03cc-cbf1-4493-aa35-0721241e3f28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:42:58.987962 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:42:58.987890    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-serving-cert: secret "maas-api-e2e-inf-b-bdee41-serving-cert" not found
Jul 30 13:42:58.987962 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:42:58.987941    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/536d03cc-cbf1-4493-aa35-0721241e3f28-maas-api-tls podName:536d03cc-cbf1-4493-aa35-0721241e3f28 nodeName:}" failed. No retries permitted until 2026-07-30 13:42:59.487925623 +0000 UTC m=+2602.760389997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/536d03cc-cbf1-4493-aa35-0721241e3f28-maas-api-tls") pod "maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n" (UID: "536d03cc-cbf1-4493-aa35-0721241e3f28") : secret "maas-api-e2e-inf-b-bdee41-serving-cert" not found
Jul 30 13:42:58.988494 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.988472    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/536d03cc-cbf1-4493-aa35-0721241e3f28-openshift-service-ca\") pod \"maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n\" (UID: \"536d03cc-cbf1-4493-aa35-0721241e3f28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:42:58.997226 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:58.997200    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss5jt\" (UniqueName: \"kubernetes.io/projected/536d03cc-cbf1-4493-aa35-0721241e3f28-kube-api-access-ss5jt\") pod \"maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n\" (UID: \"536d03cc-cbf1-4493-aa35-0721241e3f28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:42:59.492238 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:59.492193    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/536d03cc-cbf1-4493-aa35-0721241e3f28-maas-api-tls\") pod \"maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n\" (UID: \"536d03cc-cbf1-4493-aa35-0721241e3f28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:42:59.494751 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:59.494727    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/536d03cc-cbf1-4493-aa35-0721241e3f28-maas-api-tls\") pod \"maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n\" (UID: \"536d03cc-cbf1-4493-aa35-0721241e3f28\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:42:59.667801 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:59.667765    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:42:59.804940 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:59.804891    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"]
Jul 30 13:42:59.807674 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:42:59.807581    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536d03cc_cbf1_4493_aa35_0721241e3f28.slice/crio-2a6ebd4c1b9853f91206d171601e9146adc31c318694ff369ce5402052563878 WatchSource:0}: Error finding container 2a6ebd4c1b9853f91206d171601e9146adc31c318694ff369ce5402052563878: Status 404 returned error can't find the container with id 2a6ebd4c1b9853f91206d171601e9146adc31c318694ff369ce5402052563878
Jul 30 13:42:59.809048 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:42:59.809025    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 13:43:00.767578 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:00.767538    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n" event={"ID":"536d03cc-cbf1-4493-aa35-0721241e3f28","Type":"ContainerStarted","Data":"2a6ebd4c1b9853f91206d171601e9146adc31c318694ff369ce5402052563878"}
Jul 30 13:43:01.773105 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:01.773067    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n" event={"ID":"536d03cc-cbf1-4493-aa35-0721241e3f28","Type":"ContainerStarted","Data":"38d5483c158dbc074b9db5d3fe9a56c99a815e158edf95be35ac621c56169809"}
Jul 30 13:43:01.773557 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:01.773153    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:43:01.793175 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:01.793127    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n" podStartSLOduration=2.3145276 podStartE2EDuration="3.793099582s" totalImagesPullingTime="1.478571982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:42:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:43:01.791356265 +0000 UTC m=+2605.063820683" watchObservedRunningTime="2026-07-30 13:43:01.793099582 +0000 UTC m=+2605.065564018"
Jul 30 13:43:03.139295 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:03.139255    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:43:07.789252 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:07.789221    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:43:11.747778 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:11.747745    2557 patch_prober.go:28] interesting pod/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 30 13:43:11.748188 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:11.747801    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x" podUID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 30 13:43:11.828100 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:11.828068    2557 generic.go:356] "Generic (PLEG): container finished" podID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" containerID="f196e0fd62625ef17e99161ada6eaaf2c37c7ef64344706a4172824378fb322a" exitCode=0
Jul 30 13:43:11.828254 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:11.828142    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x" event={"ID":"ba5192ec-9b5e-4603-a72d-d6206a2a1823","Type":"ContainerDied","Data":"f196e0fd62625ef17e99161ada6eaaf2c37c7ef64344706a4172824378fb322a"}
Jul 30 13:43:11.828572 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:11.828555    2557 scope.go:117] "RemoveContainer" containerID="f196e0fd62625ef17e99161ada6eaaf2c37c7ef64344706a4172824378fb322a"
Jul 30 13:43:11.829590 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:11.829565    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:43:11.829681 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:11.829648    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:43:11.829722 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:11.829681    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:43:12.834890 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:12.834852    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x" event={"ID":"ba5192ec-9b5e-4603-a72d-d6206a2a1823","Type":"ContainerStarted","Data":"939904a396b33522525f8e8b49207fe4c6c0724b067dc07576737c63f92eb2fe"}
Jul 30 13:43:13.533020 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:13.532927    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:43:13.533211 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:13.533187    2557 patch_prober.go:28] interesting pod/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 30 13:43:13.533258 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:13.533241    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x" podUID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 30 13:43:14.532436 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:14.532400    2557 patch_prober.go:28] interesting pod/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 30 13:43:14.532873 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:14.532454    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x" podUID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 30 13:43:15.532223 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:15.532188    2557 patch_prober.go:28] interesting pod/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 30 13:43:15.532386 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:15.532246    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x" podUID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 30 13:43:16.532520 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:16.532483    2557 patch_prober.go:28] interesting pod/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 30 13:43:16.532920 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:16.532551    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x" podUID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 30 13:43:17.536546 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:17.536511    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:43:17.537041 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:17.536808    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:43:17.537550 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:17.537532    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:43:41.755900 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.755863    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"]
Jul 30 13:43:41.764021 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.763994    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.768640 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.768613    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-628283\"/\"default-dockercfg-mnzb9\""
Jul 30 13:43:41.768640 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.768618    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-628283\"/\"kube-root-ca.crt\""
Jul 30 13:43:41.768852 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.768623    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-628283\"/\"test-model-628283-kserve-self-signed-certs\""
Jul 30 13:43:41.768852 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.768672    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-628283\"/\"openshift-service-ca.crt\""
Jul 30 13:43:41.772940 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.772922    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"]
Jul 30 13:43:41.876010 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.875973    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsw2d\" (UniqueName: \"kubernetes.io/projected/ce72f108-5509-4cfb-b2af-22b70e652056-kube-api-access-qsw2d\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.876186 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.876034    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce72f108-5509-4cfb-b2af-22b70e652056-tls-certs\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.876186 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.876114    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-dshm\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.876186 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.876178    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-model-cache\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.876296 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.876201    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-tmp-dir\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.876296 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.876223    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-home\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.976731 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.976699    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsw2d\" (UniqueName: \"kubernetes.io/projected/ce72f108-5509-4cfb-b2af-22b70e652056-kube-api-access-qsw2d\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.976921 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.976761    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce72f108-5509-4cfb-b2af-22b70e652056-tls-certs\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.976921 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.976790    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-dshm\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.977060 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.976932    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-model-cache\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.977060 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.976955    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-tmp-dir\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.977060 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.976988    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-home\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.977372 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.977350    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-model-cache\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.977491 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.977353    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-tmp-dir\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.977491 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.977428    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-home\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.979193 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.979170    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-dshm\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.979283 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.979260    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce72f108-5509-4cfb-b2af-22b70e652056-tls-certs\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:41.986262 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:41.986240    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsw2d\" (UniqueName: \"kubernetes.io/projected/ce72f108-5509-4cfb-b2af-22b70e652056-kube-api-access-qsw2d\") pod \"test-model-628283-kserve-df7bf5975-9z68m\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") " pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:42.075002 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:42.074924    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:42.209149 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:42.209122    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"]
Jul 30 13:43:42.211365 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:43:42.211339    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce72f108_5509_4cfb_b2af_22b70e652056.slice/crio-ab183f7fe6b87fb9c39be119acbbe270f1722c314ce398528804377da31f5cd6 WatchSource:0}: Error finding container ab183f7fe6b87fb9c39be119acbbe270f1722c314ce398528804377da31f5cd6: Status 404 returned error can't find the container with id ab183f7fe6b87fb9c39be119acbbe270f1722c314ce398528804377da31f5cd6
Jul 30 13:43:42.972001 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:42.971965    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m" event={"ID":"ce72f108-5509-4cfb-b2af-22b70e652056","Type":"ContainerStarted","Data":"824cff51014144426d33a67322935ae4d912796175ab60aa4327d38b1f22e542"}
Jul 30 13:43:42.972429 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:42.972010    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m" event={"ID":"ce72f108-5509-4cfb-b2af-22b70e652056","Type":"ContainerStarted","Data":"ab183f7fe6b87fb9c39be119acbbe270f1722c314ce398528804377da31f5cd6"}
Jul 30 13:43:42.991291 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:42.991213    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m" podStartSLOduration=1.9911965820000002 podStartE2EDuration="1.991196582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:43:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:43:42.989541877 +0000 UTC m=+2646.262006275" watchObservedRunningTime="2026-07-30 13:43:42.991196582 +0000 UTC m=+2646.263660979"
Jul 30 13:43:46.753558 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:46.753524    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:43:52.075568 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:52.075533    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:52.076008 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:52.075617    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:52.087937 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:52.087912    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:53.029918 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:53.029888    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:43:58.349205 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.349171    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"]
Jul 30 13:43:58.352560 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.352544    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.356626 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.356568    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-bdee41\"/\"openshift-service-ca.crt\""
Jul 30 13:43:58.356756 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.356699    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-bdee41\"/\"test-model-bdee41-kserve-self-signed-certs\""
Jul 30 13:43:58.356756 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.356744    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-bdee41\"/\"default-dockercfg-pgcd6\""
Jul 30 13:43:58.356872 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.356853    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-bdee41\"/\"kube-root-ca.crt\""
Jul 30 13:43:58.364587 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.364567    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"]
Jul 30 13:43:58.532820 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.532791    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64324f26-b57f-4ae4-aab8-89d14ad9025e-tls-certs\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.532995 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.532835    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl7hc\" (UniqueName: \"kubernetes.io/projected/64324f26-b57f-4ae4-aab8-89d14ad9025e-kube-api-access-kl7hc\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.532995 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.532863    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-home\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.532995 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.532891    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-model-cache\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.532995 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.532961    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-tmp-dir\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.533136 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.533008    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-dshm\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.634458 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.634369    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64324f26-b57f-4ae4-aab8-89d14ad9025e-tls-certs\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.634458 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.634422    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl7hc\" (UniqueName: \"kubernetes.io/projected/64324f26-b57f-4ae4-aab8-89d14ad9025e-kube-api-access-kl7hc\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.634458 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.634445    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-home\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.634775 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.634470    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-model-cache\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.634775 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.634515    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-tmp-dir\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.634775 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.634550    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-dshm\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.635006 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.634985    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-home\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.635066 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.635011    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-tmp-dir\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.635066 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.635053    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-model-cache\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.636859 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.636833    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-dshm\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.637147 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.637127    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64324f26-b57f-4ae4-aab8-89d14ad9025e-tls-certs\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.643996 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.643972    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl7hc\" (UniqueName: \"kubernetes.io/projected/64324f26-b57f-4ae4-aab8-89d14ad9025e-kube-api-access-kl7hc\") pod \"test-model-bdee41-kserve-6654875f95-5nljw\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") " pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.662938 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.662914    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:43:58.796649 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:58.796491    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"]
Jul 30 13:43:58.798800 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:43:58.798776    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64324f26_b57f_4ae4_aab8_89d14ad9025e.slice/crio-307c4171bdd122ad01d6d1083f23327dbbaed45c6ff59648bb3fb38ef4589ced WatchSource:0}: Error finding container 307c4171bdd122ad01d6d1083f23327dbbaed45c6ff59648bb3fb38ef4589ced: Status 404 returned error can't find the container with id 307c4171bdd122ad01d6d1083f23327dbbaed45c6ff59648bb3fb38ef4589ced
Jul 30 13:43:59.048530 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:59.048494    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw" event={"ID":"64324f26-b57f-4ae4-aab8-89d14ad9025e","Type":"ContainerStarted","Data":"b02387bfd1234b9469a9dd325eed51d2e7c3a9ffb1b88261e6c68bcefe807a31"}
Jul 30 13:43:59.048530 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:59.048537    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw" event={"ID":"64324f26-b57f-4ae4-aab8-89d14ad9025e","Type":"ContainerStarted","Data":"307c4171bdd122ad01d6d1083f23327dbbaed45c6ff59648bb3fb38ef4589ced"}
Jul 30 13:43:59.069550 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:43:59.069502    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw" podStartSLOduration=1.069487282 podStartE2EDuration="1.069487282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:43:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:43:59.06778359 +0000 UTC m=+2662.340247979" watchObservedRunningTime="2026-07-30 13:43:59.069487282 +0000 UTC m=+2662.341951679"
Jul 30 13:44:03.748077 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:03.748038    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:44:08.663360 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:08.663318    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:44:08.663802 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:08.663375    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:44:08.675741 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:08.675716    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:44:09.105625 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:09.105525    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:44:42.639711 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:42.639680    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"]
Jul 30 13:44:43.548361 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:43.548326    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"]
Jul 30 13:44:47.251639 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:47.251590    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:44:57.658274 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.658213    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m" podUID="ce72f108-5509-4cfb-b2af-22b70e652056" containerName="main" containerID="cri-o://824cff51014144426d33a67322935ae4d912796175ab60aa4327d38b1f22e542" gracePeriod=45
Jul 30 13:44:57.912676 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.912620    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:44:57.973072 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.973041    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce72f108-5509-4cfb-b2af-22b70e652056-tls-certs\") pod \"ce72f108-5509-4cfb-b2af-22b70e652056\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") "
Jul 30 13:44:57.973222 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.973080    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-dshm\") pod \"ce72f108-5509-4cfb-b2af-22b70e652056\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") "
Jul 30 13:44:57.973222 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.973118    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-model-cache\") pod \"ce72f108-5509-4cfb-b2af-22b70e652056\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") "
Jul 30 13:44:57.973222 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.973139    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-home\") pod \"ce72f108-5509-4cfb-b2af-22b70e652056\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") "
Jul 30 13:44:57.973222 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.973175    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsw2d\" (UniqueName: \"kubernetes.io/projected/ce72f108-5509-4cfb-b2af-22b70e652056-kube-api-access-qsw2d\") pod \"ce72f108-5509-4cfb-b2af-22b70e652056\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") "
Jul 30 13:44:57.973222 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.973212    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-tmp-dir\") pod \"ce72f108-5509-4cfb-b2af-22b70e652056\" (UID: \"ce72f108-5509-4cfb-b2af-22b70e652056\") "
Jul 30 13:44:57.973488 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.973382    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-model-cache" (OuterVolumeSpecName: "model-cache") pod "ce72f108-5509-4cfb-b2af-22b70e652056" (UID: "ce72f108-5509-4cfb-b2af-22b70e652056"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:44:57.973488 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.973420    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-home" (OuterVolumeSpecName: "home") pod "ce72f108-5509-4cfb-b2af-22b70e652056" (UID: "ce72f108-5509-4cfb-b2af-22b70e652056"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:44:57.973585 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.973498    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ce72f108-5509-4cfb-b2af-22b70e652056" (UID: "ce72f108-5509-4cfb-b2af-22b70e652056"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:44:57.973585 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.973525    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-model-cache\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:44:57.973585 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.973538    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-home\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:44:57.975312 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.975288    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-dshm" (OuterVolumeSpecName: "dshm") pod "ce72f108-5509-4cfb-b2af-22b70e652056" (UID: "ce72f108-5509-4cfb-b2af-22b70e652056"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:44:57.975421 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.975316    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce72f108-5509-4cfb-b2af-22b70e652056-kube-api-access-qsw2d" (OuterVolumeSpecName: "kube-api-access-qsw2d") pod "ce72f108-5509-4cfb-b2af-22b70e652056" (UID: "ce72f108-5509-4cfb-b2af-22b70e652056"). InnerVolumeSpecName "kube-api-access-qsw2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:44:57.975421 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:57.975340    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce72f108-5509-4cfb-b2af-22b70e652056-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ce72f108-5509-4cfb-b2af-22b70e652056" (UID: "ce72f108-5509-4cfb-b2af-22b70e652056"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 13:44:58.074522 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.074490    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce72f108-5509-4cfb-b2af-22b70e652056-tls-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:44:58.074522 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.074514    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-dshm\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:44:58.074522 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.074523    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsw2d\" (UniqueName: \"kubernetes.io/projected/ce72f108-5509-4cfb-b2af-22b70e652056-kube-api-access-qsw2d\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:44:58.074522 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.074532    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce72f108-5509-4cfb-b2af-22b70e652056-tmp-dir\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:44:58.331244 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.331150    2557 generic.go:356] "Generic (PLEG): container finished" podID="ce72f108-5509-4cfb-b2af-22b70e652056" containerID="824cff51014144426d33a67322935ae4d912796175ab60aa4327d38b1f22e542" exitCode=0
Jul 30 13:44:58.331244 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.331189    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m" event={"ID":"ce72f108-5509-4cfb-b2af-22b70e652056","Type":"ContainerDied","Data":"824cff51014144426d33a67322935ae4d912796175ab60aa4327d38b1f22e542"}
Jul 30 13:44:58.331244 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.331199    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"
Jul 30 13:44:58.331244 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.331220    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m" event={"ID":"ce72f108-5509-4cfb-b2af-22b70e652056","Type":"ContainerDied","Data":"ab183f7fe6b87fb9c39be119acbbe270f1722c314ce398528804377da31f5cd6"}
Jul 30 13:44:58.331244 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.331240    2557 scope.go:117] "RemoveContainer" containerID="824cff51014144426d33a67322935ae4d912796175ab60aa4327d38b1f22e542"
Jul 30 13:44:58.342073 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.342051    2557 scope.go:117] "RemoveContainer" containerID="824cff51014144426d33a67322935ae4d912796175ab60aa4327d38b1f22e542"
Jul 30 13:44:58.343189 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:44:58.343161    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"824cff51014144426d33a67322935ae4d912796175ab60aa4327d38b1f22e542\": container with ID starting with 824cff51014144426d33a67322935ae4d912796175ab60aa4327d38b1f22e542 not found: ID does not exist" containerID="824cff51014144426d33a67322935ae4d912796175ab60aa4327d38b1f22e542"
Jul 30 13:44:58.343292 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.343197    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824cff51014144426d33a67322935ae4d912796175ab60aa4327d38b1f22e542"} err="failed to get container status \"824cff51014144426d33a67322935ae4d912796175ab60aa4327d38b1f22e542\": rpc error: code = NotFound desc = could not find container \"824cff51014144426d33a67322935ae4d912796175ab60aa4327d38b1f22e542\": container with ID starting with 824cff51014144426d33a67322935ae4d912796175ab60aa4327d38b1f22e542 not found: ID does not exist"
Jul 30 13:44:58.353259 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.353236    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"]
Jul 30 13:44:58.357610 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.357574    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-628283/test-model-628283-kserve-df7bf5975-9z68m"]
Jul 30 13:44:58.567237 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.567195    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw" podUID="64324f26-b57f-4ae4-aab8-89d14ad9025e" containerName="main" containerID="cri-o://b02387bfd1234b9469a9dd325eed51d2e7c3a9ffb1b88261e6c68bcefe807a31" gracePeriod=45
Jul 30 13:44:58.814975 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.814953    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:44:58.883557 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.883527    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-tmp-dir\") pod \"64324f26-b57f-4ae4-aab8-89d14ad9025e\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") "
Jul 30 13:44:58.883761 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.883566    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-dshm\") pod \"64324f26-b57f-4ae4-aab8-89d14ad9025e\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") "
Jul 30 13:44:58.883761 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.883609    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64324f26-b57f-4ae4-aab8-89d14ad9025e-tls-certs\") pod \"64324f26-b57f-4ae4-aab8-89d14ad9025e\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") "
Jul 30 13:44:58.883761 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.883640    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-model-cache\") pod \"64324f26-b57f-4ae4-aab8-89d14ad9025e\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") "
Jul 30 13:44:58.883761 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.883704    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-home\") pod \"64324f26-b57f-4ae4-aab8-89d14ad9025e\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") "
Jul 30 13:44:58.883761 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.883746    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kl7hc\" (UniqueName: \"kubernetes.io/projected/64324f26-b57f-4ae4-aab8-89d14ad9025e-kube-api-access-kl7hc\") pod \"64324f26-b57f-4ae4-aab8-89d14ad9025e\" (UID: \"64324f26-b57f-4ae4-aab8-89d14ad9025e\") "
Jul 30 13:44:58.884042 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.883877    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "64324f26-b57f-4ae4-aab8-89d14ad9025e" (UID: "64324f26-b57f-4ae4-aab8-89d14ad9025e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:44:58.884042 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.883917    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-model-cache" (OuterVolumeSpecName: "model-cache") pod "64324f26-b57f-4ae4-aab8-89d14ad9025e" (UID: "64324f26-b57f-4ae4-aab8-89d14ad9025e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:44:58.884042 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.884019    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-home" (OuterVolumeSpecName: "home") pod "64324f26-b57f-4ae4-aab8-89d14ad9025e" (UID: "64324f26-b57f-4ae4-aab8-89d14ad9025e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:44:58.884158 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.884035    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-tmp-dir\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:44:58.884158 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.884076    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-model-cache\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:44:58.885877 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.885854    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-dshm" (OuterVolumeSpecName: "dshm") pod "64324f26-b57f-4ae4-aab8-89d14ad9025e" (UID: "64324f26-b57f-4ae4-aab8-89d14ad9025e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:44:58.885991 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.885953    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64324f26-b57f-4ae4-aab8-89d14ad9025e-kube-api-access-kl7hc" (OuterVolumeSpecName: "kube-api-access-kl7hc") pod "64324f26-b57f-4ae4-aab8-89d14ad9025e" (UID: "64324f26-b57f-4ae4-aab8-89d14ad9025e"). InnerVolumeSpecName "kube-api-access-kl7hc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:44:58.885991 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.885954    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64324f26-b57f-4ae4-aab8-89d14ad9025e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "64324f26-b57f-4ae4-aab8-89d14ad9025e" (UID: "64324f26-b57f-4ae4-aab8-89d14ad9025e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 13:44:58.984828 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.984736    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-dshm\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:44:58.984828 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.984763    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64324f26-b57f-4ae4-aab8-89d14ad9025e-tls-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:44:58.984828 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.984776    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64324f26-b57f-4ae4-aab8-89d14ad9025e-home\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:44:58.984828 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:58.984787    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl7hc\" (UniqueName: \"kubernetes.io/projected/64324f26-b57f-4ae4-aab8-89d14ad9025e-kube-api-access-kl7hc\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:44:59.183533 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:59.183494    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce72f108-5509-4cfb-b2af-22b70e652056" path="/var/lib/kubelet/pods/ce72f108-5509-4cfb-b2af-22b70e652056/volumes"
Jul 30 13:44:59.338324 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:59.338228    2557 generic.go:356] "Generic (PLEG): container finished" podID="64324f26-b57f-4ae4-aab8-89d14ad9025e" containerID="b02387bfd1234b9469a9dd325eed51d2e7c3a9ffb1b88261e6c68bcefe807a31" exitCode=0
Jul 30 13:44:59.338324 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:59.338271    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"
Jul 30 13:44:59.338324 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:59.338286    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw" event={"ID":"64324f26-b57f-4ae4-aab8-89d14ad9025e","Type":"ContainerDied","Data":"b02387bfd1234b9469a9dd325eed51d2e7c3a9ffb1b88261e6c68bcefe807a31"}
Jul 30 13:44:59.338324 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:59.338323    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw" event={"ID":"64324f26-b57f-4ae4-aab8-89d14ad9025e","Type":"ContainerDied","Data":"307c4171bdd122ad01d6d1083f23327dbbaed45c6ff59648bb3fb38ef4589ced"}
Jul 30 13:44:59.338645 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:59.338345    2557 scope.go:117] "RemoveContainer" containerID="b02387bfd1234b9469a9dd325eed51d2e7c3a9ffb1b88261e6c68bcefe807a31"
Jul 30 13:44:59.346719 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:59.346702    2557 scope.go:117] "RemoveContainer" containerID="b02387bfd1234b9469a9dd325eed51d2e7c3a9ffb1b88261e6c68bcefe807a31"
Jul 30 13:44:59.347012 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:44:59.346995    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b02387bfd1234b9469a9dd325eed51d2e7c3a9ffb1b88261e6c68bcefe807a31\": container with ID starting with b02387bfd1234b9469a9dd325eed51d2e7c3a9ffb1b88261e6c68bcefe807a31 not found: ID does not exist" containerID="b02387bfd1234b9469a9dd325eed51d2e7c3a9ffb1b88261e6c68bcefe807a31"
Jul 30 13:44:59.347068 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:59.347020    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b02387bfd1234b9469a9dd325eed51d2e7c3a9ffb1b88261e6c68bcefe807a31"} err="failed to get container status \"b02387bfd1234b9469a9dd325eed51d2e7c3a9ffb1b88261e6c68bcefe807a31\": rpc error: code = NotFound desc = could not find container \"b02387bfd1234b9469a9dd325eed51d2e7c3a9ffb1b88261e6c68bcefe807a31\": container with ID starting with b02387bfd1234b9469a9dd325eed51d2e7c3a9ffb1b88261e6c68bcefe807a31 not found: ID does not exist"
Jul 30 13:44:59.355881 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:59.355857    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"]
Jul 30 13:44:59.359414 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:44:59.359385    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-bdee41/test-model-bdee41-kserve-6654875f95-5nljw"]
Jul 30 13:45:00.136505 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:00.136470    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756985-ljc9r"]
Jul 30 13:45:00.136956 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:00.136830    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64324f26-b57f-4ae4-aab8-89d14ad9025e" containerName="main"
Jul 30 13:45:00.136956 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:00.136841    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="64324f26-b57f-4ae4-aab8-89d14ad9025e" containerName="main"
Jul 30 13:45:00.136956 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:00.136861    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce72f108-5509-4cfb-b2af-22b70e652056" containerName="main"
Jul 30 13:45:00.136956 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:00.136867    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce72f108-5509-4cfb-b2af-22b70e652056" containerName="main"
Jul 30 13:45:00.136956 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:00.136919    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce72f108-5509-4cfb-b2af-22b70e652056" containerName="main"
Jul 30 13:45:00.136956 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:00.136931    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="64324f26-b57f-4ae4-aab8-89d14ad9025e" containerName="main"
Jul 30 13:45:00.141260 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:00.141241    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756985-ljc9r"
Jul 30 13:45:00.144119 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:00.144101    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxdpz\""
Jul 30 13:45:00.148611 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:00.148255    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756985-ljc9r"]
Jul 30 13:45:00.152657 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:00.152636    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756985-ljc9r"
Jul 30 13:45:00.281636 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:00.281606    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756985-ljc9r"]
Jul 30 13:45:00.284877 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:45:00.284854    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82579774_d905_4280_8886_bb44ea4570d4.slice/crio-34d2c27c8d4a1489b60b1996e33e7f7cae5a8d0834c1e467b30bfa42b8d9a3d7 WatchSource:0}: Error finding container 34d2c27c8d4a1489b60b1996e33e7f7cae5a8d0834c1e467b30bfa42b8d9a3d7: Status 404 returned error can't find the container with id 34d2c27c8d4a1489b60b1996e33e7f7cae5a8d0834c1e467b30bfa42b8d9a3d7
Jul 30 13:45:00.345280 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:00.345250    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756985-ljc9r" event={"ID":"82579774-d905-4280-8886-bb44ea4570d4","Type":"ContainerStarted","Data":"34d2c27c8d4a1489b60b1996e33e7f7cae5a8d0834c1e467b30bfa42b8d9a3d7"}
Jul 30 13:45:01.183775 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:01.183749    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64324f26-b57f-4ae4-aab8-89d14ad9025e" path="/var/lib/kubelet/pods/64324f26-b57f-4ae4-aab8-89d14ad9025e/volumes"
Jul 30 13:45:01.351887 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:01.351854    2557 generic.go:356] "Generic (PLEG): container finished" podID="82579774-d905-4280-8886-bb44ea4570d4" containerID="55083138bc68d3e491e78093983b10978775c18e1e0dc7ec4bcaf5ae92356f04" exitCode=0
Jul 30 13:45:01.352059 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:01.351930    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756985-ljc9r" event={"ID":"82579774-d905-4280-8886-bb44ea4570d4","Type":"ContainerDied","Data":"55083138bc68d3e491e78093983b10978775c18e1e0dc7ec4bcaf5ae92356f04"}
Jul 30 13:45:01.413147 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:01.413117    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"]
Jul 30 13:45:01.413334 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:01.413313    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj" podUID="d152f512-99ac-4365-b565-b775b22a374d" containerName="istio-proxy" containerID="cri-o://755a8e05901d4a0f754edd6bc5bb39af24a1802105d64a95a51ec17a5bf39a3c" gracePeriod=30
Jul 30 13:45:01.788260 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:01.788217    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g"]
Jul 30 13:45:01.792197 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:01.792180    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g"
Jul 30 13:45:01.802791 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:01.802759    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g"]
Jul 30 13:45:01.910836 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:01.910799    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16b3f8f4-79e1-479d-9cc5-6e7f12593d1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g\" (UID: \"16b3f8f4-79e1-479d-9cc5-6e7f12593d1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g"
Jul 30 13:45:02.012378 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:02.012336    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16b3f8f4-79e1-479d-9cc5-6e7f12593d1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g\" (UID: \"16b3f8f4-79e1-479d-9cc5-6e7f12593d1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g"
Jul 30 13:45:02.012984 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:02.012964    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16b3f8f4-79e1-479d-9cc5-6e7f12593d1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g\" (UID: \"16b3f8f4-79e1-479d-9cc5-6e7f12593d1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g"
Jul 30 13:45:02.112905 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:02.112799    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g"
Jul 30 13:45:02.246435 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:02.246407    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g"]
Jul 30 13:45:02.248669 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:45:02.248631    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b3f8f4_79e1_479d_9cc5_6e7f12593d1e.slice/crio-d73eccdc246c40796db0aecc75bca5808584ef8a236f049a30215f4b1f52df76 WatchSource:0}: Error finding container d73eccdc246c40796db0aecc75bca5808584ef8a236f049a30215f4b1f52df76: Status 404 returned error can't find the container with id d73eccdc246c40796db0aecc75bca5808584ef8a236f049a30215f4b1f52df76
Jul 30 13:45:02.358187 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:02.358146    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g" event={"ID":"16b3f8f4-79e1-479d-9cc5-6e7f12593d1e","Type":"ContainerStarted","Data":"d73eccdc246c40796db0aecc75bca5808584ef8a236f049a30215f4b1f52df76"}
Jul 30 13:45:02.501292 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:02.501256    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756985-ljc9r"
Jul 30 13:45:03.363785 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:03.363704    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756985-ljc9r"
Jul 30 13:45:03.364210 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:03.363703    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756985-ljc9r" event={"ID":"82579774-d905-4280-8886-bb44ea4570d4","Type":"ContainerDied","Data":"34d2c27c8d4a1489b60b1996e33e7f7cae5a8d0834c1e467b30bfa42b8d9a3d7"}
Jul 30 13:45:03.364210 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:03.363822    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34d2c27c8d4a1489b60b1996e33e7f7cae5a8d0834c1e467b30bfa42b8d9a3d7"
Jul 30 13:45:03.365428 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:03.365398    2557 generic.go:356] "Generic (PLEG): container finished" podID="16b3f8f4-79e1-479d-9cc5-6e7f12593d1e" containerID="6a9a3b2319d85b09d59db16ee85802d4364db4526238b15cfb1372c6bba92dee" exitCode=0
Jul 30 13:45:03.365571 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:03.365426    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g" event={"ID":"16b3f8f4-79e1-479d-9cc5-6e7f12593d1e","Type":"ContainerDied","Data":"6a9a3b2319d85b09d59db16ee85802d4364db4526238b15cfb1372c6bba92dee"}
Jul 30 13:45:03.922093 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:03.922055    2557 patch_prober.go:28] interesting pod/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 13:45:03.922264 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:03.922119    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj" podUID="d152f512-99ac-4365-b565-b775b22a374d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 13:45:04.339583 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:04.339503    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:45:04.500332 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:04.500307    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g"
Jul 30 13:45:04.638901 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:04.638807    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16b3f8f4-79e1-479d-9cc5-6e7f12593d1e-maas-api-service-ca\") pod \"16b3f8f4-79e1-479d-9cc5-6e7f12593d1e\" (UID: \"16b3f8f4-79e1-479d-9cc5-6e7f12593d1e\") "
Jul 30 13:45:04.639220 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:04.639199    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16b3f8f4-79e1-479d-9cc5-6e7f12593d1e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "16b3f8f4-79e1-479d-9cc5-6e7f12593d1e" (UID: "16b3f8f4-79e1-479d-9cc5-6e7f12593d1e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:45:04.740465 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:04.740428    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16b3f8f4-79e1-479d-9cc5-6e7f12593d1e-maas-api-service-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:05.376699 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:05.376667    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g"
Jul 30 13:45:05.376873 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:05.376666    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g" event={"ID":"16b3f8f4-79e1-479d-9cc5-6e7f12593d1e","Type":"ContainerDied","Data":"d73eccdc246c40796db0aecc75bca5808584ef8a236f049a30215f4b1f52df76"}
Jul 30 13:45:05.376873 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:05.376804    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d73eccdc246c40796db0aecc75bca5808584ef8a236f049a30215f4b1f52df76"
Jul 30 13:45:06.678977 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.678919    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:45:06.760878 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.760844    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-credential-socket\") pod \"d152f512-99ac-4365-b565-b775b22a374d\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") "
Jul 30 13:45:06.761070 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.760887    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-istio-envoy\") pod \"d152f512-99ac-4365-b565-b775b22a374d\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") "
Jul 30 13:45:06.761070 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.760936    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-workload-socket\") pod \"d152f512-99ac-4365-b565-b775b22a374d\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") "
Jul 30 13:45:06.761070 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.760965    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d152f512-99ac-4365-b565-b775b22a374d-istiod-ca-cert\") pod \"d152f512-99ac-4365-b565-b775b22a374d\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") "
Jul 30 13:45:06.761070 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.761023    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d152f512-99ac-4365-b565-b775b22a374d-istio-token\") pod \"d152f512-99ac-4365-b565-b775b22a374d\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") "
Jul 30 13:45:06.761070 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.761051    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d152f512-99ac-4365-b565-b775b22a374d-istio-podinfo\") pod \"d152f512-99ac-4365-b565-b775b22a374d\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") "
Jul 30 13:45:06.761326 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.761091    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-workload-certs\") pod \"d152f512-99ac-4365-b565-b775b22a374d\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") "
Jul 30 13:45:06.761326 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.761128    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbq58\" (UniqueName: \"kubernetes.io/projected/d152f512-99ac-4365-b565-b775b22a374d-kube-api-access-sbq58\") pod \"d152f512-99ac-4365-b565-b775b22a374d\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") "
Jul 30 13:45:06.761326 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.761140    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d152f512-99ac-4365-b565-b775b22a374d" (UID: "d152f512-99ac-4365-b565-b775b22a374d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:45:06.761326 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.761209    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-istio-data\") pod \"d152f512-99ac-4365-b565-b775b22a374d\" (UID: \"d152f512-99ac-4365-b565-b775b22a374d\") "
Jul 30 13:45:06.761526 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.761372    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d152f512-99ac-4365-b565-b775b22a374d" (UID: "d152f512-99ac-4365-b565-b775b22a374d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:45:06.761526 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.761440    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d152f512-99ac-4365-b565-b775b22a374d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d152f512-99ac-4365-b565-b775b22a374d" (UID: "d152f512-99ac-4365-b565-b775b22a374d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:45:06.761526 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.761513    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-workload-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:06.761725 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.761562    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d152f512-99ac-4365-b565-b775b22a374d" (UID: "d152f512-99ac-4365-b565-b775b22a374d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:45:06.761725 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.761572    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-credential-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:06.762104 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.762081    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-istio-data" (OuterVolumeSpecName: "istio-data") pod "d152f512-99ac-4365-b565-b775b22a374d" (UID: "d152f512-99ac-4365-b565-b775b22a374d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:45:06.763154 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.763126    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d152f512-99ac-4365-b565-b775b22a374d-istio-token" (OuterVolumeSpecName: "istio-token") pod "d152f512-99ac-4365-b565-b775b22a374d" (UID: "d152f512-99ac-4365-b565-b775b22a374d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:45:06.763154 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.763145    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d152f512-99ac-4365-b565-b775b22a374d" (UID: "d152f512-99ac-4365-b565-b775b22a374d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:45:06.763521 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.763496    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d152f512-99ac-4365-b565-b775b22a374d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d152f512-99ac-4365-b565-b775b22a374d" (UID: "d152f512-99ac-4365-b565-b775b22a374d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 13:45:06.763521 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.763512    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d152f512-99ac-4365-b565-b775b22a374d-kube-api-access-sbq58" (OuterVolumeSpecName: "kube-api-access-sbq58") pod "d152f512-99ac-4365-b565-b775b22a374d" (UID: "d152f512-99ac-4365-b565-b775b22a374d"). InnerVolumeSpecName "kube-api-access-sbq58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:45:06.862037 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.861996    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-workload-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:06.862037 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.862029    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d152f512-99ac-4365-b565-b775b22a374d-istiod-ca-cert\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:06.862037 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.862038    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d152f512-99ac-4365-b565-b775b22a374d-istio-token\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:06.862037 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.862046    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d152f512-99ac-4365-b565-b775b22a374d-istio-podinfo\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:06.862306 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.862054    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbq58\" (UniqueName: \"kubernetes.io/projected/d152f512-99ac-4365-b565-b775b22a374d-kube-api-access-sbq58\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:06.862306 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.862064    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-istio-data\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:06.862306 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:06.862072    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d152f512-99ac-4365-b565-b775b22a374d-istio-envoy\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:07.388736 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:07.388697    2557 generic.go:356] "Generic (PLEG): container finished" podID="d152f512-99ac-4365-b565-b775b22a374d" containerID="755a8e05901d4a0f754edd6bc5bb39af24a1802105d64a95a51ec17a5bf39a3c" exitCode=0
Jul 30 13:45:07.388920 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:07.388766    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"
Jul 30 13:45:07.388920 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:07.388778    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj" event={"ID":"d152f512-99ac-4365-b565-b775b22a374d","Type":"ContainerDied","Data":"755a8e05901d4a0f754edd6bc5bb39af24a1802105d64a95a51ec17a5bf39a3c"}
Jul 30 13:45:07.388920 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:07.388841    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj" event={"ID":"d152f512-99ac-4365-b565-b775b22a374d","Type":"ContainerDied","Data":"6b1c1ea4ccb35be513c5b2829eb5615ddc1093d4fb57ca9ddf0cfcc6a5ae89a1"}
Jul 30 13:45:07.388920 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:07.388865    2557 scope.go:117] "RemoveContainer" containerID="755a8e05901d4a0f754edd6bc5bb39af24a1802105d64a95a51ec17a5bf39a3c"
Jul 30 13:45:07.397806 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:07.397783    2557 scope.go:117] "RemoveContainer" containerID="cabb5bd281da027a046361fb51418805ed0a6f255fee1e34a879d19f52c756bf"
Jul 30 13:45:07.405826 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:07.405804    2557 scope.go:117] "RemoveContainer" containerID="755a8e05901d4a0f754edd6bc5bb39af24a1802105d64a95a51ec17a5bf39a3c"
Jul 30 13:45:07.406328 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:45:07.406173    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"755a8e05901d4a0f754edd6bc5bb39af24a1802105d64a95a51ec17a5bf39a3c\": container with ID starting with 755a8e05901d4a0f754edd6bc5bb39af24a1802105d64a95a51ec17a5bf39a3c not found: ID does not exist" containerID="755a8e05901d4a0f754edd6bc5bb39af24a1802105d64a95a51ec17a5bf39a3c"
Jul 30 13:45:07.406428 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:07.406342    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"755a8e05901d4a0f754edd6bc5bb39af24a1802105d64a95a51ec17a5bf39a3c"} err="failed to get container status \"755a8e05901d4a0f754edd6bc5bb39af24a1802105d64a95a51ec17a5bf39a3c\": rpc error: code = NotFound desc = could not find container \"755a8e05901d4a0f754edd6bc5bb39af24a1802105d64a95a51ec17a5bf39a3c\": container with ID starting with 755a8e05901d4a0f754edd6bc5bb39af24a1802105d64a95a51ec17a5bf39a3c not found: ID does not exist"
Jul 30 13:45:07.406428 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:07.406368    2557 scope.go:117] "RemoveContainer" containerID="cabb5bd281da027a046361fb51418805ed0a6f255fee1e34a879d19f52c756bf"
Jul 30 13:45:07.406675 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:45:07.406651    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cabb5bd281da027a046361fb51418805ed0a6f255fee1e34a879d19f52c756bf\": container with ID starting with cabb5bd281da027a046361fb51418805ed0a6f255fee1e34a879d19f52c756bf not found: ID does not exist" containerID="cabb5bd281da027a046361fb51418805ed0a6f255fee1e34a879d19f52c756bf"
Jul 30 13:45:07.406725 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:07.406685    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cabb5bd281da027a046361fb51418805ed0a6f255fee1e34a879d19f52c756bf"} err="failed to get container status \"cabb5bd281da027a046361fb51418805ed0a6f255fee1e34a879d19f52c756bf\": rpc error: code = NotFound desc = could not find container \"cabb5bd281da027a046361fb51418805ed0a6f255fee1e34a879d19f52c756bf\": container with ID starting with cabb5bd281da027a046361fb51418805ed0a6f255fee1e34a879d19f52c756bf not found: ID does not exist"
Jul 30 13:45:07.408426 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:07.408407    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"]
Jul 30 13:45:07.412112 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:07.412092    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-628283-openshift-default-55d778fc4d-cb2qj"]
Jul 30 13:45:09.183610 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:09.183573    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d152f512-99ac-4365-b565-b775b22a374d" path="/var/lib/kubelet/pods/d152f512-99ac-4365-b565-b775b22a374d/volumes"
Jul 30 13:45:12.343000 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:12.342964    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"]
Jul 30 13:45:12.344483 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:12.343168    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n" podUID="536d03cc-cbf1-4493-aa35-0721241e3f28" containerName="maas-api" containerID="cri-o://38d5483c158dbc074b9db5d3fe9a56c99a815e158edf95be35ac621c56169809" gracePeriod=30
Jul 30 13:45:12.535895 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:12.535860    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g"]
Jul 30 13:45:12.537527 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:12.537500    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bdee41-c5j9g"]
Jul 30 13:45:12.804329 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:12.804305    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:45:12.914915 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:12.914886    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/536d03cc-cbf1-4493-aa35-0721241e3f28-maas-api-tls\") pod \"536d03cc-cbf1-4493-aa35-0721241e3f28\" (UID: \"536d03cc-cbf1-4493-aa35-0721241e3f28\") "
Jul 30 13:45:12.915107 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:12.914932    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss5jt\" (UniqueName: \"kubernetes.io/projected/536d03cc-cbf1-4493-aa35-0721241e3f28-kube-api-access-ss5jt\") pod \"536d03cc-cbf1-4493-aa35-0721241e3f28\" (UID: \"536d03cc-cbf1-4493-aa35-0721241e3f28\") "
Jul 30 13:45:12.915107 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:12.914968    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/536d03cc-cbf1-4493-aa35-0721241e3f28-openshift-service-ca\") pod \"536d03cc-cbf1-4493-aa35-0721241e3f28\" (UID: \"536d03cc-cbf1-4493-aa35-0721241e3f28\") "
Jul 30 13:45:12.915409 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:12.915384    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/536d03cc-cbf1-4493-aa35-0721241e3f28-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "536d03cc-cbf1-4493-aa35-0721241e3f28" (UID: "536d03cc-cbf1-4493-aa35-0721241e3f28"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:45:12.917155 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:12.917125    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/536d03cc-cbf1-4493-aa35-0721241e3f28-kube-api-access-ss5jt" (OuterVolumeSpecName: "kube-api-access-ss5jt") pod "536d03cc-cbf1-4493-aa35-0721241e3f28" (UID: "536d03cc-cbf1-4493-aa35-0721241e3f28"). InnerVolumeSpecName "kube-api-access-ss5jt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:45:12.917237 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:12.917196    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/536d03cc-cbf1-4493-aa35-0721241e3f28-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "536d03cc-cbf1-4493-aa35-0721241e3f28" (UID: "536d03cc-cbf1-4493-aa35-0721241e3f28"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 13:45:13.015746 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:13.015701    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/536d03cc-cbf1-4493-aa35-0721241e3f28-maas-api-tls\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:13.015746 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:13.015739    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss5jt\" (UniqueName: \"kubernetes.io/projected/536d03cc-cbf1-4493-aa35-0721241e3f28-kube-api-access-ss5jt\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:13.015746 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:13.015753    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/536d03cc-cbf1-4493-aa35-0721241e3f28-openshift-service-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:13.183957 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:13.183879    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16b3f8f4-79e1-479d-9cc5-6e7f12593d1e" path="/var/lib/kubelet/pods/16b3f8f4-79e1-479d-9cc5-6e7f12593d1e/volumes"
Jul 30 13:45:13.420750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:13.420717    2557 generic.go:356] "Generic (PLEG): container finished" podID="536d03cc-cbf1-4493-aa35-0721241e3f28" containerID="38d5483c158dbc074b9db5d3fe9a56c99a815e158edf95be35ac621c56169809" exitCode=0
Jul 30 13:45:13.421277 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:13.420762    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"
Jul 30 13:45:13.421277 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:13.420795    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n" event={"ID":"536d03cc-cbf1-4493-aa35-0721241e3f28","Type":"ContainerDied","Data":"38d5483c158dbc074b9db5d3fe9a56c99a815e158edf95be35ac621c56169809"}
Jul 30 13:45:13.421277 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:13.420850    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n" event={"ID":"536d03cc-cbf1-4493-aa35-0721241e3f28","Type":"ContainerDied","Data":"2a6ebd4c1b9853f91206d171601e9146adc31c318694ff369ce5402052563878"}
Jul 30 13:45:13.421277 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:13.420872    2557 scope.go:117] "RemoveContainer" containerID="38d5483c158dbc074b9db5d3fe9a56c99a815e158edf95be35ac621c56169809"
Jul 30 13:45:13.434991 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:13.434969    2557 scope.go:117] "RemoveContainer" containerID="38d5483c158dbc074b9db5d3fe9a56c99a815e158edf95be35ac621c56169809"
Jul 30 13:45:13.435289 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:45:13.435272    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38d5483c158dbc074b9db5d3fe9a56c99a815e158edf95be35ac621c56169809\": container with ID starting with 38d5483c158dbc074b9db5d3fe9a56c99a815e158edf95be35ac621c56169809 not found: ID does not exist" containerID="38d5483c158dbc074b9db5d3fe9a56c99a815e158edf95be35ac621c56169809"
Jul 30 13:45:13.435336 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:13.435296    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38d5483c158dbc074b9db5d3fe9a56c99a815e158edf95be35ac621c56169809"} err="failed to get container status \"38d5483c158dbc074b9db5d3fe9a56c99a815e158edf95be35ac621c56169809\": rpc error: code = NotFound desc = could not find container \"38d5483c158dbc074b9db5d3fe9a56c99a815e158edf95be35ac621c56169809\": container with ID starting with 38d5483c158dbc074b9db5d3fe9a56c99a815e158edf95be35ac621c56169809 not found: ID does not exist"
Jul 30 13:45:13.442586 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:13.442561    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"]
Jul 30 13:45:13.445772 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:13.445750    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n"]
Jul 30 13:45:15.183803 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:15.183774    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="536d03cc-cbf1-4493-aa35-0721241e3f28" path="/var/lib/kubelet/pods/536d03cc-cbf1-4493-aa35-0721241e3f28/volumes"
Jul 30 13:45:15.537138 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:15.537057    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:45:15.780906 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:15.780860    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdee41-968cf974c-kvp4n" podUID="536d03cc-cbf1-4493-aa35-0721241e3f28" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.44:8443/health\": context deadline exceeded"
Jul 30 13:45:19.015880 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.015844    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"]
Jul 30 13:45:19.016321 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.016090    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x" podUID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" containerName="istio-proxy" containerID="cri-o://939904a396b33522525f8e8b49207fe4c6c0724b067dc07576737c63f92eb2fe" gracePeriod=30
Jul 30 13:45:19.922453 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.922412    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"]
Jul 30 13:45:19.922932 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.922891    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="536d03cc-cbf1-4493-aa35-0721241e3f28" containerName="maas-api"
Jul 30 13:45:19.922932 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.922930    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="536d03cc-cbf1-4493-aa35-0721241e3f28" containerName="maas-api"
Jul 30 13:45:19.923129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.922945    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d152f512-99ac-4365-b565-b775b22a374d" containerName="istio-proxy"
Jul 30 13:45:19.923129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.922951    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d152f512-99ac-4365-b565-b775b22a374d" containerName="istio-proxy"
Jul 30 13:45:19.923129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.922970    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82579774-d905-4280-8886-bb44ea4570d4" containerName="cleanup"
Jul 30 13:45:19.923129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.922993    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="82579774-d905-4280-8886-bb44ea4570d4" containerName="cleanup"
Jul 30 13:45:19.923129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.923013    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16b3f8f4-79e1-479d-9cc5-6e7f12593d1e" containerName="revoke-keys"
Jul 30 13:45:19.923129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.923023    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="16b3f8f4-79e1-479d-9cc5-6e7f12593d1e" containerName="revoke-keys"
Jul 30 13:45:19.923129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.923090    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="16b3f8f4-79e1-479d-9cc5-6e7f12593d1e" containerName="revoke-keys"
Jul 30 13:45:19.923129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.923103    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d152f512-99ac-4365-b565-b775b22a374d" containerName="istio-proxy"
Jul 30 13:45:19.923129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.923113    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="536d03cc-cbf1-4493-aa35-0721241e3f28" containerName="maas-api"
Jul 30 13:45:19.923129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.923119    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d152f512-99ac-4365-b565-b775b22a374d" containerName="istio-proxy"
Jul 30 13:45:19.923129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.923125    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="82579774-d905-4280-8886-bb44ea4570d4" containerName="cleanup"
Jul 30 13:45:19.926031 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.926012    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:19.928838 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.928815    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-45e018-openshift-default-dockercfg-dzgnh\""
Jul 30 13:45:19.939991 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:19.939970    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"]
Jul 30 13:45:20.079057 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.079023    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-workload-socket\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.079547 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.079067    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-credential-socket\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.079547 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.079098    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr8fq\" (UniqueName: \"kubernetes.io/projected/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-kube-api-access-jr8fq\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.079547 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.079192    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-podinfo\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.079547 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.079264    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-workload-certs\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.079547 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.079302    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-data\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.079547 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.079354    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istiod-ca-cert\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.079547 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.079380    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-token\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.079547 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.079446    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-envoy\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.180143 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.180057    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-envoy\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.180143 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.180096    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-workload-socket\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.180143 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.180126    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-credential-socket\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.180376 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.180246    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr8fq\" (UniqueName: \"kubernetes.io/projected/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-kube-api-access-jr8fq\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.180376 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.180313    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-podinfo\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.180376 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.180355    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-workload-certs\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.180529 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.180390    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-data\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.180529 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.180480    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-credential-socket\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.180529 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.180509    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-workload-socket\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.180529 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.180521    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istiod-ca-cert\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.180777 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.180558    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-token\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.180777 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.180716    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-data\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.180777 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.180746    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-workload-certs\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.181138 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.181121    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istiod-ca-cert\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.182400 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.182383    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-envoy\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.182801 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.182778    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-podinfo\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.188608 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.188572    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-token\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.189041 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.189023    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr8fq\" (UniqueName: \"kubernetes.io/projected/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-kube-api-access-jr8fq\") pod \"e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") " pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.236497 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.236462    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:20.372073 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.372046    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"]
Jul 30 13:45:20.374492 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:45:20.374457    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd12f3a8d_dbbb_4a3f_8b45_3968e51dc4bf.slice/crio-9b99358cab45c35f96f74cac76c8d436d2bdbd8104a80b6e09966e3737dc1631 WatchSource:0}: Error finding container 9b99358cab45c35f96f74cac76c8d436d2bdbd8104a80b6e09966e3737dc1631: Status 404 returned error can't find the container with id 9b99358cab45c35f96f74cac76c8d436d2bdbd8104a80b6e09966e3737dc1631
Jul 30 13:45:20.376583 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.376550    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:45:20.376683 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.376632    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:45:20.376683 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.376662    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:45:20.457079 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.456999    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f" event={"ID":"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf","Type":"ContainerStarted","Data":"b7cecb529b9566a27adf0333188a53250d93e65ddbfc8697e837a91422a875ea"}
Jul 30 13:45:20.457079 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.457068    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f" event={"ID":"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf","Type":"ContainerStarted","Data":"9b99358cab45c35f96f74cac76c8d436d2bdbd8104a80b6e09966e3737dc1631"}
Jul 30 13:45:20.479381 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:20.479324    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f" podStartSLOduration=1.479302085 podStartE2EDuration="1.479302085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:45:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:45:20.476157839 +0000 UTC m=+2743.748622269" watchObservedRunningTime="2026-07-30 13:45:20.479302085 +0000 UTC m=+2743.751766483"
Jul 30 13:45:21.237651 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:21.237621    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:21.242586 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:21.242564    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:21.461510 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:21.461485    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:21.462642 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:21.462623    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:23.511167 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.511127    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"]
Jul 30 13:45:23.511727 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.511707    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d152f512-99ac-4365-b565-b775b22a374d" containerName="istio-proxy"
Jul 30 13:45:23.511782 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.511730    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d152f512-99ac-4365-b565-b775b22a374d" containerName="istio-proxy"
Jul 30 13:45:23.515215 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.515191    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:45:23.518851 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.518828    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 13:45:23.518978 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.518848    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-45e018-serving-cert\""
Jul 30 13:45:23.518978 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.518842    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 13:45:23.519944 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.519922    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dnpqq\""
Jul 30 13:45:23.525183 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.525164    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"]
Jul 30 13:45:23.611114 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.611080    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b0dabe5-c75d-4d23-991a-f474e0663bae-openshift-service-ca\") pod \"maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg\" (UID: \"1b0dabe5-c75d-4d23-991a-f474e0663bae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:45:23.611484 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.611133    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmfps\" (UniqueName: \"kubernetes.io/projected/1b0dabe5-c75d-4d23-991a-f474e0663bae-kube-api-access-pmfps\") pod \"maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg\" (UID: \"1b0dabe5-c75d-4d23-991a-f474e0663bae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:45:23.611484 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.611181    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b0dabe5-c75d-4d23-991a-f474e0663bae-maas-api-tls\") pod \"maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg\" (UID: \"1b0dabe5-c75d-4d23-991a-f474e0663bae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:45:23.712829 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.712795    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b0dabe5-c75d-4d23-991a-f474e0663bae-openshift-service-ca\") pod \"maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg\" (UID: \"1b0dabe5-c75d-4d23-991a-f474e0663bae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:45:23.712984 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.712850    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmfps\" (UniqueName: \"kubernetes.io/projected/1b0dabe5-c75d-4d23-991a-f474e0663bae-kube-api-access-pmfps\") pod \"maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg\" (UID: \"1b0dabe5-c75d-4d23-991a-f474e0663bae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:45:23.712984 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.712892    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b0dabe5-c75d-4d23-991a-f474e0663bae-maas-api-tls\") pod \"maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg\" (UID: \"1b0dabe5-c75d-4d23-991a-f474e0663bae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:45:23.713389 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.713365    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b0dabe5-c75d-4d23-991a-f474e0663bae-openshift-service-ca\") pod \"maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg\" (UID: \"1b0dabe5-c75d-4d23-991a-f474e0663bae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:45:23.715216 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.715193    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b0dabe5-c75d-4d23-991a-f474e0663bae-maas-api-tls\") pod \"maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg\" (UID: \"1b0dabe5-c75d-4d23-991a-f474e0663bae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:45:23.722412 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.722391    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmfps\" (UniqueName: \"kubernetes.io/projected/1b0dabe5-c75d-4d23-991a-f474e0663bae-kube-api-access-pmfps\") pod \"maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg\" (UID: \"1b0dabe5-c75d-4d23-991a-f474e0663bae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:45:23.826919 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.826842    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:45:23.975325 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:23.975302    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"]
Jul 30 13:45:23.977043 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:45:23.977009    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b0dabe5_c75d_4d23_991a_f474e0663bae.slice/crio-ae2712d0b4a4eec4c63995a34e1d1d4d8ce133ba02861c19392b54a02f18c974 WatchSource:0}: Error finding container ae2712d0b4a4eec4c63995a34e1d1d4d8ce133ba02861c19392b54a02f18c974: Status 404 returned error can't find the container with id ae2712d0b4a4eec4c63995a34e1d1d4d8ce133ba02861c19392b54a02f18c974
Jul 30 13:45:24.255352 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.255327    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:45:24.420304 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.420279    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-token\") pod \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") "
Jul 30 13:45:24.420407 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.420319    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istiod-ca-cert\") pod \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") "
Jul 30 13:45:24.420407 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.420349    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-credential-socket\") pod \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") "
Jul 30 13:45:24.420407 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.420374    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlfvd\" (UniqueName: \"kubernetes.io/projected/ba5192ec-9b5e-4603-a72d-d6206a2a1823-kube-api-access-wlfvd\") pod \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") "
Jul 30 13:45:24.420584 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.420406    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-podinfo\") pod \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") "
Jul 30 13:45:24.420584 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.420470    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-envoy\") pod \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") "
Jul 30 13:45:24.420584 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.420498    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-workload-certs\") pod \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") "
Jul 30 13:45:24.420584 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.420526    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-data\") pod \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") "
Jul 30 13:45:24.420584 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.420581    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-workload-socket\") pod \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\" (UID: \"ba5192ec-9b5e-4603-a72d-d6206a2a1823\") "
Jul 30 13:45:24.420877 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.420661    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba5192ec-9b5e-4603-a72d-d6206a2a1823" (UID: "ba5192ec-9b5e-4603-a72d-d6206a2a1823"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:45:24.420877 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.420713    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba5192ec-9b5e-4603-a72d-d6206a2a1823" (UID: "ba5192ec-9b5e-4603-a72d-d6206a2a1823"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:45:24.420998 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.420858    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba5192ec-9b5e-4603-a72d-d6206a2a1823" (UID: "ba5192ec-9b5e-4603-a72d-d6206a2a1823"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:45:24.420998 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.420924    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba5192ec-9b5e-4603-a72d-d6206a2a1823" (UID: "ba5192ec-9b5e-4603-a72d-d6206a2a1823"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:45:24.421106 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.421036    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-workload-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:24.421106 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.421049    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-workload-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:24.421106 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.421062    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istiod-ca-cert\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:24.421106 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.421076    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-credential-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:24.421624 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.421568    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba5192ec-9b5e-4603-a72d-d6206a2a1823" (UID: "ba5192ec-9b5e-4603-a72d-d6206a2a1823"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:45:24.422844 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.422825    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba5192ec-9b5e-4603-a72d-d6206a2a1823" (UID: "ba5192ec-9b5e-4603-a72d-d6206a2a1823"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:45:24.423158 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.423129    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba5192ec-9b5e-4603-a72d-d6206a2a1823" (UID: "ba5192ec-9b5e-4603-a72d-d6206a2a1823"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 13:45:24.423240 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.423140    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba5192ec-9b5e-4603-a72d-d6206a2a1823-kube-api-access-wlfvd" (OuterVolumeSpecName: "kube-api-access-wlfvd") pod "ba5192ec-9b5e-4603-a72d-d6206a2a1823" (UID: "ba5192ec-9b5e-4603-a72d-d6206a2a1823"). InnerVolumeSpecName "kube-api-access-wlfvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:45:24.423515 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.423492    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba5192ec-9b5e-4603-a72d-d6206a2a1823" (UID: "ba5192ec-9b5e-4603-a72d-d6206a2a1823"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:45:24.477610 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.477558    2557 generic.go:356] "Generic (PLEG): container finished" podID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" containerID="939904a396b33522525f8e8b49207fe4c6c0724b067dc07576737c63f92eb2fe" exitCode=0
Jul 30 13:45:24.477724 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.477640    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"
Jul 30 13:45:24.477724 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.477639    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x" event={"ID":"ba5192ec-9b5e-4603-a72d-d6206a2a1823","Type":"ContainerDied","Data":"939904a396b33522525f8e8b49207fe4c6c0724b067dc07576737c63f92eb2fe"}
Jul 30 13:45:24.477838 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.477727    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x" event={"ID":"ba5192ec-9b5e-4603-a72d-d6206a2a1823","Type":"ContainerDied","Data":"971323c930fe67a1e504907d16163b984a7edab04ac61f81d83a4dca99950416"}
Jul 30 13:45:24.477838 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.477758    2557 scope.go:117] "RemoveContainer" containerID="939904a396b33522525f8e8b49207fe4c6c0724b067dc07576737c63f92eb2fe"
Jul 30 13:45:24.479519 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.479418    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg" event={"ID":"1b0dabe5-c75d-4d23-991a-f474e0663bae","Type":"ContainerStarted","Data":"305f52f98539db753d03c5483c5a78673d3c75b00d1cb587f2a243dcf59f6bb0"}
Jul 30 13:45:24.479519 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.479464    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg" event={"ID":"1b0dabe5-c75d-4d23-991a-f474e0663bae","Type":"ContainerStarted","Data":"ae2712d0b4a4eec4c63995a34e1d1d4d8ce133ba02861c19392b54a02f18c974"}
Jul 30 13:45:24.479719 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.479574    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:45:24.489262 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.489242    2557 scope.go:117] "RemoveContainer" containerID="f196e0fd62625ef17e99161ada6eaaf2c37c7ef64344706a4172824378fb322a"
Jul 30 13:45:24.497544 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.497524    2557 scope.go:117] "RemoveContainer" containerID="939904a396b33522525f8e8b49207fe4c6c0724b067dc07576737c63f92eb2fe"
Jul 30 13:45:24.497813 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:45:24.497795    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"939904a396b33522525f8e8b49207fe4c6c0724b067dc07576737c63f92eb2fe\": container with ID starting with 939904a396b33522525f8e8b49207fe4c6c0724b067dc07576737c63f92eb2fe not found: ID does not exist" containerID="939904a396b33522525f8e8b49207fe4c6c0724b067dc07576737c63f92eb2fe"
Jul 30 13:45:24.497866 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.497822    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"939904a396b33522525f8e8b49207fe4c6c0724b067dc07576737c63f92eb2fe"} err="failed to get container status \"939904a396b33522525f8e8b49207fe4c6c0724b067dc07576737c63f92eb2fe\": rpc error: code = NotFound desc = could not find container \"939904a396b33522525f8e8b49207fe4c6c0724b067dc07576737c63f92eb2fe\": container with ID starting with 939904a396b33522525f8e8b49207fe4c6c0724b067dc07576737c63f92eb2fe not found: ID does not exist"
Jul 30 13:45:24.497866 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.497840    2557 scope.go:117] "RemoveContainer" containerID="f196e0fd62625ef17e99161ada6eaaf2c37c7ef64344706a4172824378fb322a"
Jul 30 13:45:24.498083 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:45:24.498063    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f196e0fd62625ef17e99161ada6eaaf2c37c7ef64344706a4172824378fb322a\": container with ID starting with f196e0fd62625ef17e99161ada6eaaf2c37c7ef64344706a4172824378fb322a not found: ID does not exist" containerID="f196e0fd62625ef17e99161ada6eaaf2c37c7ef64344706a4172824378fb322a"
Jul 30 13:45:24.498131 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.498089    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f196e0fd62625ef17e99161ada6eaaf2c37c7ef64344706a4172824378fb322a"} err="failed to get container status \"f196e0fd62625ef17e99161ada6eaaf2c37c7ef64344706a4172824378fb322a\": rpc error: code = NotFound desc = could not find container \"f196e0fd62625ef17e99161ada6eaaf2c37c7ef64344706a4172824378fb322a\": container with ID starting with f196e0fd62625ef17e99161ada6eaaf2c37c7ef64344706a4172824378fb322a not found: ID does not exist"
Jul 30 13:45:24.500446 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.500405    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg" podStartSLOduration=1.061815924 podStartE2EDuration="1.5003925s" totalImagesPullingTime="438.576576ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:45:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:45:24.499885267 +0000 UTC m=+2747.772349667" watchObservedRunningTime="2026-07-30 13:45:24.5003925 +0000 UTC m=+2747.772856887"
Jul 30 13:45:24.514729 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.514707    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"]
Jul 30 13:45:24.518761 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.518738    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-bdee41-openshift-default-65b74cc98-cb95x"]
Jul 30 13:45:24.522238 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.522220    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-envoy\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:24.522306 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.522239    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-data\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:24.522306 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.522249    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-token\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:24.522306 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.522278    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlfvd\" (UniqueName: \"kubernetes.io/projected/ba5192ec-9b5e-4603-a72d-d6206a2a1823-kube-api-access-wlfvd\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:24.522306 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:24.522287    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba5192ec-9b5e-4603-a72d-d6206a2a1823-istio-podinfo\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:45:25.183495 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:25.183462    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" path="/var/lib/kubelet/pods/ba5192ec-9b5e-4603-a72d-d6206a2a1823/volumes"
Jul 30 13:45:27.639966 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:27.639929    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:45:30.493251 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:30.493222    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:45:35.537062 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:35.536981    2557 generic.go:356] "Generic (PLEG): container finished" podID="d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" containerID="b7cecb529b9566a27adf0333188a53250d93e65ddbfc8697e837a91422a875ea" exitCode=0
Jul 30 13:45:35.537062 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:35.537046    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f" event={"ID":"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf","Type":"ContainerDied","Data":"b7cecb529b9566a27adf0333188a53250d93e65ddbfc8697e837a91422a875ea"}
Jul 30 13:45:35.537496 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:35.537408    2557 scope.go:117] "RemoveContainer" containerID="b7cecb529b9566a27adf0333188a53250d93e65ddbfc8697e837a91422a875ea"
Jul 30 13:45:35.538403 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:35.538373    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:45:35.538469 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:35.538456    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:45:35.538532 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:35.538516    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:45:36.461888 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:36.461842    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:36.543741 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:36.543706    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f" event={"ID":"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf","Type":"ContainerStarted","Data":"820bb6b7f79df654c067b4eb2ecbc7b980357b19a15f67e6c49280f53653f26d"}
Jul 30 13:45:37.236912 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:37.236877    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:37.238063 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:37.238035    2557 patch_prober.go:28] interesting pod/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 30 13:45:37.238194 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:37.238094    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f" podUID="d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 30 13:45:38.236865 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:38.236832    2557 patch_prober.go:28] interesting pod/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 30 13:45:38.237269 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:38.236886    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f" podUID="d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 30 13:45:39.237497 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:39.237462    2557 patch_prober.go:28] interesting pod/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 30 13:45:39.237915 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:39.237516    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f" podUID="d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 30 13:45:40.236906 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:40.236869    2557 patch_prober.go:28] interesting pod/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 30 13:45:40.237159 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:40.236922    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f" podUID="d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 30 13:45:41.241395 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:41.241366    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:41.241851 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:41.241635    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:41.242328 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:41.242308    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:45:48.280045 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.280014    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"]
Jul 30 13:45:48.280452 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.280389    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" containerName="istio-proxy"
Jul 30 13:45:48.280452 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.280400    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" containerName="istio-proxy"
Jul 30 13:45:48.280528 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.280475    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" containerName="istio-proxy"
Jul 30 13:45:48.280528 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.280483    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" containerName="istio-proxy"
Jul 30 13:45:48.283377 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.283361    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.286526 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.286503    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-5fdbfb-openshift-default-dockercfg-xfvcd\""
Jul 30 13:45:48.301466 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.301438    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"]
Jul 30 13:45:48.331278 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.331246    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-podinfo\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.331412 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.331290    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-workload-certs\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.331412 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.331316    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-token\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.331412 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.331340    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjzwh\" (UniqueName: \"kubernetes.io/projected/d4ad391a-c17d-4252-8cc4-666963b43c54-kube-api-access-kjzwh\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.331412 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.331388    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-workload-socket\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.331549 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.331419    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4ad391a-c17d-4252-8cc4-666963b43c54-istiod-ca-cert\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.331549 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.331446    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-envoy\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.331549 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.331513    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-credential-socket\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.331549 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.331532    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-data\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.432570 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.432536    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-envoy\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.432570 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.432575    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-credential-socket\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.432805 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.432611    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-data\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.432805 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.432654    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-podinfo\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.432805 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.432703    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-workload-certs\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.432805 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.432740    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-token\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.432805 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.432761    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjzwh\" (UniqueName: \"kubernetes.io/projected/d4ad391a-c17d-4252-8cc4-666963b43c54-kube-api-access-kjzwh\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.432805 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.432795    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-workload-socket\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.433106 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.432839    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4ad391a-c17d-4252-8cc4-666963b43c54-istiod-ca-cert\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.433106 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.433089    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-data\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.433228 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.433123    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-credential-socket\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.433427 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.433400    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-workload-socket\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.433817 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.433675    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4ad391a-c17d-4252-8cc4-666963b43c54-istiod-ca-cert\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.433817 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.433326    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-workload-certs\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.435113 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.435094    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-envoy\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.435347 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.435328    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-podinfo\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.444154 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.444133    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-token\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.444576 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.444560    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjzwh\" (UniqueName: \"kubernetes.io/projected/d4ad391a-c17d-4252-8cc4-666963b43c54-kube-api-access-kjzwh\") pod \"e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") " pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.593567 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.593487    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:48.729141 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.729116    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"]
Jul 30 13:45:48.730854 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:45:48.730827    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4ad391a_c17d_4252_8cc4_666963b43c54.slice/crio-e0f6c7fc5ea406d7c8ca3024c606bca26b83f51a71211920d5e243fa0d728ac9 WatchSource:0}: Error finding container e0f6c7fc5ea406d7c8ca3024c606bca26b83f51a71211920d5e243fa0d728ac9: Status 404 returned error can't find the container with id e0f6c7fc5ea406d7c8ca3024c606bca26b83f51a71211920d5e243fa0d728ac9
Jul 30 13:45:48.733102 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.733073    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:45:48.733195 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.733138    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:45:48.733195 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:48.733165    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:45:49.608229 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:49.608192    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl" event={"ID":"d4ad391a-c17d-4252-8cc4-666963b43c54","Type":"ContainerStarted","Data":"d10863e6805b3d5fda4351f4f0a84a166266da1700b30d620447dc32c5265462"}
Jul 30 13:45:49.608642 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:49.608245    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl" event={"ID":"d4ad391a-c17d-4252-8cc4-666963b43c54","Type":"ContainerStarted","Data":"e0f6c7fc5ea406d7c8ca3024c606bca26b83f51a71211920d5e243fa0d728ac9"}
Jul 30 13:45:49.632780 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:49.632730    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl" podStartSLOduration=1.632716132 podStartE2EDuration="1.632716132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:45:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:45:49.630352364 +0000 UTC m=+2772.902816782" watchObservedRunningTime="2026-07-30 13:45:49.632716132 +0000 UTC m=+2772.905180529"
Jul 30 13:45:50.594247 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:50.594210    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:50.599260 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:50.599236    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:50.612575 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:50.612548    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:45:50.613487 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:45:50.613468    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:46:00.931311 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:00.931223    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:46:09.713941 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:09.713907    2557 generic.go:356] "Generic (PLEG): container finished" podID="d4ad391a-c17d-4252-8cc4-666963b43c54" containerID="d10863e6805b3d5fda4351f4f0a84a166266da1700b30d620447dc32c5265462" exitCode=0
Jul 30 13:46:09.713941 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:09.713946    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl" event={"ID":"d4ad391a-c17d-4252-8cc4-666963b43c54","Type":"ContainerDied","Data":"d10863e6805b3d5fda4351f4f0a84a166266da1700b30d620447dc32c5265462"}
Jul 30 13:46:09.714438 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:09.714307    2557 scope.go:117] "RemoveContainer" containerID="d10863e6805b3d5fda4351f4f0a84a166266da1700b30d620447dc32c5265462"
Jul 30 13:46:09.715354 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:09.715327    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:46:09.715419 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:09.715393    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:46:09.715456 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:09.715429    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:46:10.721575 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:10.721543    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl" event={"ID":"d4ad391a-c17d-4252-8cc4-666963b43c54","Type":"ContainerStarted","Data":"5b77a988dafd79592e99bc463b1dc8576fd40cfa46ca4f07496f0af5d782a3d3"}
Jul 30 13:46:11.593935 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:11.593899    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:46:11.594268 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:11.594207    2557 patch_prober.go:28] interesting pod/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 30 13:46:11.594346 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:11.594264    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl" podUID="d4ad391a-c17d-4252-8cc4-666963b43c54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 30 13:46:12.594808 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:12.594766    2557 patch_prober.go:28] interesting pod/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 30 13:46:12.595220 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:12.594827    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl" podUID="d4ad391a-c17d-4252-8cc4-666963b43c54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 30 13:46:13.594010 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:13.593970    2557 patch_prober.go:28] interesting pod/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 30 13:46:13.594211 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:13.594030    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl" podUID="d4ad391a-c17d-4252-8cc4-666963b43c54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 30 13:46:14.593983 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:14.593939    2557 patch_prober.go:28] interesting pod/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 30 13:46:14.594381 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:14.594012    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl" podUID="d4ad391a-c17d-4252-8cc4-666963b43c54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 30 13:46:15.597766 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:15.597734    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:46:15.598211 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:15.598101    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:46:15.598740 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:15.598721    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:46:35.003695 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.003660    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"]
Jul 30 13:46:35.004267 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.003916    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg" podUID="1b0dabe5-c75d-4d23-991a-f474e0663bae" containerName="maas-api" containerID="cri-o://305f52f98539db753d03c5483c5a78673d3c75b00d1cb587f2a243dcf59f6bb0" gracePeriod=30
Jul 30 13:46:35.257805 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.257739    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:46:35.376634 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.376575    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmfps\" (UniqueName: \"kubernetes.io/projected/1b0dabe5-c75d-4d23-991a-f474e0663bae-kube-api-access-pmfps\") pod \"1b0dabe5-c75d-4d23-991a-f474e0663bae\" (UID: \"1b0dabe5-c75d-4d23-991a-f474e0663bae\") "
Jul 30 13:46:35.376830 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.376653    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b0dabe5-c75d-4d23-991a-f474e0663bae-openshift-service-ca\") pod \"1b0dabe5-c75d-4d23-991a-f474e0663bae\" (UID: \"1b0dabe5-c75d-4d23-991a-f474e0663bae\") "
Jul 30 13:46:35.376830 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.376670    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b0dabe5-c75d-4d23-991a-f474e0663bae-maas-api-tls\") pod \"1b0dabe5-c75d-4d23-991a-f474e0663bae\" (UID: \"1b0dabe5-c75d-4d23-991a-f474e0663bae\") "
Jul 30 13:46:35.377023 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.376997    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b0dabe5-c75d-4d23-991a-f474e0663bae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1b0dabe5-c75d-4d23-991a-f474e0663bae" (UID: "1b0dabe5-c75d-4d23-991a-f474e0663bae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:46:35.378823 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.378786    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b0dabe5-c75d-4d23-991a-f474e0663bae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1b0dabe5-c75d-4d23-991a-f474e0663bae" (UID: "1b0dabe5-c75d-4d23-991a-f474e0663bae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 13:46:35.378932 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.378829    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b0dabe5-c75d-4d23-991a-f474e0663bae-kube-api-access-pmfps" (OuterVolumeSpecName: "kube-api-access-pmfps") pod "1b0dabe5-c75d-4d23-991a-f474e0663bae" (UID: "1b0dabe5-c75d-4d23-991a-f474e0663bae"). InnerVolumeSpecName "kube-api-access-pmfps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:46:35.478083 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.478052    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmfps\" (UniqueName: \"kubernetes.io/projected/1b0dabe5-c75d-4d23-991a-f474e0663bae-kube-api-access-pmfps\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:46:35.478083 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.478078    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b0dabe5-c75d-4d23-991a-f474e0663bae-openshift-service-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:46:35.478083 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.478088    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b0dabe5-c75d-4d23-991a-f474e0663bae-maas-api-tls\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:46:35.842765 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.842726    2557 generic.go:356] "Generic (PLEG): container finished" podID="1b0dabe5-c75d-4d23-991a-f474e0663bae" containerID="305f52f98539db753d03c5483c5a78673d3c75b00d1cb587f2a243dcf59f6bb0" exitCode=0
Jul 30 13:46:35.842945 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.842779    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"
Jul 30 13:46:35.842945 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.842814    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg" event={"ID":"1b0dabe5-c75d-4d23-991a-f474e0663bae","Type":"ContainerDied","Data":"305f52f98539db753d03c5483c5a78673d3c75b00d1cb587f2a243dcf59f6bb0"}
Jul 30 13:46:35.842945 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.842888    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg" event={"ID":"1b0dabe5-c75d-4d23-991a-f474e0663bae","Type":"ContainerDied","Data":"ae2712d0b4a4eec4c63995a34e1d1d4d8ce133ba02861c19392b54a02f18c974"}
Jul 30 13:46:35.842945 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.842911    2557 scope.go:117] "RemoveContainer" containerID="305f52f98539db753d03c5483c5a78673d3c75b00d1cb587f2a243dcf59f6bb0"
Jul 30 13:46:35.852731 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.852718    2557 scope.go:117] "RemoveContainer" containerID="305f52f98539db753d03c5483c5a78673d3c75b00d1cb587f2a243dcf59f6bb0"
Jul 30 13:46:35.852983 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:46:35.852960    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"305f52f98539db753d03c5483c5a78673d3c75b00d1cb587f2a243dcf59f6bb0\": container with ID starting with 305f52f98539db753d03c5483c5a78673d3c75b00d1cb587f2a243dcf59f6bb0 not found: ID does not exist" containerID="305f52f98539db753d03c5483c5a78673d3c75b00d1cb587f2a243dcf59f6bb0"
Jul 30 13:46:35.853055 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.852991    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"305f52f98539db753d03c5483c5a78673d3c75b00d1cb587f2a243dcf59f6bb0"} err="failed to get container status \"305f52f98539db753d03c5483c5a78673d3c75b00d1cb587f2a243dcf59f6bb0\": rpc error: code = NotFound desc = could not find container \"305f52f98539db753d03c5483c5a78673d3c75b00d1cb587f2a243dcf59f6bb0\": container with ID starting with 305f52f98539db753d03c5483c5a78673d3c75b00d1cb587f2a243dcf59f6bb0 not found: ID does not exist"
Jul 30 13:46:35.884185 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.884162    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"]
Jul 30 13:46:35.897966 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:35.897944    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45e018-6488df6bcc-4tjtg"]
Jul 30 13:46:37.184257 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:37.184227    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b0dabe5-c75d-4d23-991a-f474e0663bae" path="/var/lib/kubelet/pods/1b0dabe5-c75d-4d23-991a-f474e0663bae/volumes"
Jul 30 13:46:37.471822 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:37.471742    2557 scope.go:117] "RemoveContainer" containerID="d91fd5775f660aa1193f19a783ead8cae857a95979564c127ce55805a8544eaa"
Jul 30 13:46:41.239140 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:41.239104    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:46:41.579719 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:41.579634    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"]
Jul 30 13:46:41.579875 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:41.579851    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f" podUID="d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" containerName="istio-proxy" containerID="cri-o://820bb6b7f79df654c067b4eb2ecbc7b980357b19a15f67e6c49280f53653f26d" gracePeriod=30
Jul 30 13:46:46.842085 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.842023    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:46:46.894525 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.894492    2557 generic.go:356] "Generic (PLEG): container finished" podID="d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" containerID="820bb6b7f79df654c067b4eb2ecbc7b980357b19a15f67e6c49280f53653f26d" exitCode=0
Jul 30 13:46:46.894697 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.894539    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"
Jul 30 13:46:46.894697 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.894575    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f" event={"ID":"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf","Type":"ContainerDied","Data":"820bb6b7f79df654c067b4eb2ecbc7b980357b19a15f67e6c49280f53653f26d"}
Jul 30 13:46:46.894697 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.894654    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f" event={"ID":"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf","Type":"ContainerDied","Data":"9b99358cab45c35f96f74cac76c8d436d2bdbd8104a80b6e09966e3737dc1631"}
Jul 30 13:46:46.894697 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.894684    2557 scope.go:117] "RemoveContainer" containerID="820bb6b7f79df654c067b4eb2ecbc7b980357b19a15f67e6c49280f53653f26d"
Jul 30 13:46:46.903591 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.903572    2557 scope.go:117] "RemoveContainer" containerID="b7cecb529b9566a27adf0333188a53250d93e65ddbfc8697e837a91422a875ea"
Jul 30 13:46:46.911512 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.911495    2557 scope.go:117] "RemoveContainer" containerID="820bb6b7f79df654c067b4eb2ecbc7b980357b19a15f67e6c49280f53653f26d"
Jul 30 13:46:46.911776 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:46:46.911757    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"820bb6b7f79df654c067b4eb2ecbc7b980357b19a15f67e6c49280f53653f26d\": container with ID starting with 820bb6b7f79df654c067b4eb2ecbc7b980357b19a15f67e6c49280f53653f26d not found: ID does not exist" containerID="820bb6b7f79df654c067b4eb2ecbc7b980357b19a15f67e6c49280f53653f26d"
Jul 30 13:46:46.911836 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.911786    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"820bb6b7f79df654c067b4eb2ecbc7b980357b19a15f67e6c49280f53653f26d"} err="failed to get container status \"820bb6b7f79df654c067b4eb2ecbc7b980357b19a15f67e6c49280f53653f26d\": rpc error: code = NotFound desc = could not find container \"820bb6b7f79df654c067b4eb2ecbc7b980357b19a15f67e6c49280f53653f26d\": container with ID starting with 820bb6b7f79df654c067b4eb2ecbc7b980357b19a15f67e6c49280f53653f26d not found: ID does not exist"
Jul 30 13:46:46.911836 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.911804    2557 scope.go:117] "RemoveContainer" containerID="b7cecb529b9566a27adf0333188a53250d93e65ddbfc8697e837a91422a875ea"
Jul 30 13:46:46.912017 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:46:46.912003    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7cecb529b9566a27adf0333188a53250d93e65ddbfc8697e837a91422a875ea\": container with ID starting with b7cecb529b9566a27adf0333188a53250d93e65ddbfc8697e837a91422a875ea not found: ID does not exist" containerID="b7cecb529b9566a27adf0333188a53250d93e65ddbfc8697e837a91422a875ea"
Jul 30 13:46:46.912064 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.912021    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7cecb529b9566a27adf0333188a53250d93e65ddbfc8697e837a91422a875ea"} err="failed to get container status \"b7cecb529b9566a27adf0333188a53250d93e65ddbfc8697e837a91422a875ea\": rpc error: code = NotFound desc = could not find container \"b7cecb529b9566a27adf0333188a53250d93e65ddbfc8697e837a91422a875ea\": container with ID starting with b7cecb529b9566a27adf0333188a53250d93e65ddbfc8697e837a91422a875ea not found: ID does not exist"
Jul 30 13:46:46.978341 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.978298    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-credential-socket\") pod \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") "
Jul 30 13:46:46.978497 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.978382    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr8fq\" (UniqueName: \"kubernetes.io/projected/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-kube-api-access-jr8fq\") pod \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") "
Jul 30 13:46:46.978497 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.978411    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-podinfo\") pod \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") "
Jul 30 13:46:46.978497 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.978434    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-envoy\") pod \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") "
Jul 30 13:46:46.978497 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.978471    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-workload-socket\") pod \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") "
Jul 30 13:46:46.978744 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.978496    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-data\") pod \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") "
Jul 30 13:46:46.978744 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.978536    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" (UID: "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:46:46.978744 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.978563    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-workload-certs\") pod \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") "
Jul 30 13:46:46.978744 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.978649    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istiod-ca-cert\") pod \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") "
Jul 30 13:46:46.978744 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.978674    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-token\") pod \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\" (UID: \"d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf\") "
Jul 30 13:46:46.978986 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.978822    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" (UID: "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:46:46.979047 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.978983    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" (UID: "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:46:46.979047 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.979034    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-workload-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:46:46.979158 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.979055    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-credential-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:46:46.979222 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.979168    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" (UID: "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:46:46.979464 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.979441    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-data" (OuterVolumeSpecName: "istio-data") pod "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" (UID: "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:46:46.981173 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.981143    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" (UID: "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 13:46:46.981273 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.981209    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" (UID: "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:46:46.981273 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.981251    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-kube-api-access-jr8fq" (OuterVolumeSpecName: "kube-api-access-jr8fq") pod "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" (UID: "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf"). InnerVolumeSpecName "kube-api-access-jr8fq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:46:46.981351 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:46.981265    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" (UID: "d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:46:47.079755 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:47.079727    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istiod-ca-cert\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:46:47.079755 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:47.079752    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-token\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:46:47.079755 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:47.079761    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr8fq\" (UniqueName: \"kubernetes.io/projected/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-kube-api-access-jr8fq\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:46:47.079992 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:47.079773    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-podinfo\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:46:47.079992 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:47.079783    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-envoy\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:46:47.079992 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:47.079790    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-istio-data\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:46:47.079992 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:47.079797    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf-workload-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:46:47.218001 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:47.217970    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"]
Jul 30 13:46:47.223971 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:47.223937    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-45e018-openshift-default-7cfd459d8b-jqs8f"]
Jul 30 13:46:49.183126 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:46:49.183090    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" path="/var/lib/kubelet/pods/d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf/volumes"
Jul 30 13:47:02.837960 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:02.837923    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:47:04.422288 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:04.422256    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"]
Jul 30 13:47:04.422733 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:04.422465    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl" podUID="d4ad391a-c17d-4252-8cc4-666963b43c54" containerName="istio-proxy" containerID="cri-o://5b77a988dafd79592e99bc463b1dc8576fd40cfa46ca4f07496f0af5d782a3d3" gracePeriod=30
Jul 30 13:47:05.288327 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.288296    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"]
Jul 30 13:47:05.288659 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.288646    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" containerName="istio-proxy"
Jul 30 13:47:05.288659 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.288660    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba5192ec-9b5e-4603-a72d-d6206a2a1823" containerName="istio-proxy"
Jul 30 13:47:05.288750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.288677    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b0dabe5-c75d-4d23-991a-f474e0663bae" containerName="maas-api"
Jul 30 13:47:05.288750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.288683    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b0dabe5-c75d-4d23-991a-f474e0663bae" containerName="maas-api"
Jul 30 13:47:05.288750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.288695    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" containerName="istio-proxy"
Jul 30 13:47:05.288750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.288701    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" containerName="istio-proxy"
Jul 30 13:47:05.288750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.288711    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" containerName="istio-proxy"
Jul 30 13:47:05.288750 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.288716    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" containerName="istio-proxy"
Jul 30 13:47:05.288934 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.288769    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" containerName="istio-proxy"
Jul 30 13:47:05.288934 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.288777    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d12f3a8d-dbbb-4a3f-8b45-3968e51dc4bf" containerName="istio-proxy"
Jul 30 13:47:05.288934 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.288785    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b0dabe5-c75d-4d23-991a-f474e0663bae" containerName="maas-api"
Jul 30 13:47:05.293276 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.293257    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.296196 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.296180    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-f4dc0a-openshift-default-dockercfg-gd879\""
Jul 30 13:47:05.307051 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.307027    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"]
Jul 30 13:47:05.337131 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.337103    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-workload-certs\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.337131 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.337132    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-podinfo\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.337312 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.337153    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-envoy\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.337312 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.337236    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxjpw\" (UniqueName: \"kubernetes.io/projected/d1a35c54-1b67-4918-bdea-5b1a270b0a34-kube-api-access-wxjpw\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.337312 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.337280    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-data\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.337448 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.337335    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-token\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.337448 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.337392    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-credential-socket\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.337448 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.337423    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istiod-ca-cert\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.337577 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.337459    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-workload-socket\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.438200 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.438168    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-token\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.438659 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.438210    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-credential-socket\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.438659 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.438225    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istiod-ca-cert\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.438659 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.438258    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-workload-socket\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.438659 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.438285    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-workload-certs\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.438659 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.438303    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-podinfo\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.438659 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.438335    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-envoy\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.438659 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.438387    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxjpw\" (UniqueName: \"kubernetes.io/projected/d1a35c54-1b67-4918-bdea-5b1a270b0a34-kube-api-access-wxjpw\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.438659 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.438424    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-data\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.438659 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.438645    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-credential-socket\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.439095 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.438735    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-workload-socket\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.439095 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.438804    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-data\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.439095 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.438835    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-workload-certs\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.439095 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.439036    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istiod-ca-cert\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.440563 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.440534    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-envoy\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.440708 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.440690    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-podinfo\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.447173 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.447151    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-token\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.447268 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.447177    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxjpw\" (UniqueName: \"kubernetes.io/projected/d1a35c54-1b67-4918-bdea-5b1a270b0a34-kube-api-access-wxjpw\") pod \"e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") " pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.603244 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.603160    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:05.736725 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.736698    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"]
Jul 30 13:47:05.739131 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:47:05.739095    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1a35c54_1b67_4918_bdea_5b1a270b0a34.slice/crio-8b440756700dd5de25a3259f0c4821474ae970ec34decea7b9f311c2d22572fa WatchSource:0}: Error finding container 8b440756700dd5de25a3259f0c4821474ae970ec34decea7b9f311c2d22572fa: Status 404 returned error can't find the container with id 8b440756700dd5de25a3259f0c4821474ae970ec34decea7b9f311c2d22572fa
Jul 30 13:47:05.741332 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.741300    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:47:05.741417 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.741373    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:47:05.741417 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.741406    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:47:05.986644 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.986589    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc" event={"ID":"d1a35c54-1b67-4918-bdea-5b1a270b0a34","Type":"ContainerStarted","Data":"cba115ce1db2a6771bbc2a766e98a07fcd67fcea6ccf46214c92ebdb56115a34"}
Jul 30 13:47:05.986857 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:05.986697    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc" event={"ID":"d1a35c54-1b67-4918-bdea-5b1a270b0a34","Type":"ContainerStarted","Data":"8b440756700dd5de25a3259f0c4821474ae970ec34decea7b9f311c2d22572fa"}
Jul 30 13:47:06.006531 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:06.006477    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc" podStartSLOduration=1.006462584 podStartE2EDuration="1.006462584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:47:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:47:06.006057998 +0000 UTC m=+2849.278522396" watchObservedRunningTime="2026-07-30 13:47:06.006462584 +0000 UTC m=+2849.278926980"
Jul 30 13:47:06.603564 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:06.603524    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:06.608735 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:06.608708    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:06.991742 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:06.991705    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:06.992655 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:06.992635    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:09.042433 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.042399    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"]
Jul 30 13:47:09.046112 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.046089    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 13:47:09.049278 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.049251    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-f4dc0a-serving-cert\""
Jul 30 13:47:09.049407 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.049286    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 13:47:09.049407 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.049358    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 13:47:09.050616 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.050584    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dnpqq\""
Jul 30 13:47:09.057565 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.057543    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"]
Jul 30 13:47:09.068466 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.068441    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm\" (UID: \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 13:47:09.068576 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.068490    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-maas-api-tls\") pod \"maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm\" (UID: \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 13:47:09.068666 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.068648    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8dt7\" (UniqueName: \"kubernetes.io/projected/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-kube-api-access-h8dt7\") pod \"maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm\" (UID: \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 13:47:09.169896 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.169863    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8dt7\" (UniqueName: \"kubernetes.io/projected/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-kube-api-access-h8dt7\") pod \"maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm\" (UID: \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 13:47:09.169896 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.169899    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm\" (UID: \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 13:47:09.170118 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.169927    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-maas-api-tls\") pod \"maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm\" (UID: \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 13:47:09.170118 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:47:09.170020    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-serving-cert: secret "maas-api-e2e-shared-a-f4dc0a-serving-cert" not found
Jul 30 13:47:09.170118 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:47:09.170070    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-maas-api-tls podName:71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485 nodeName:}" failed. No retries permitted until 2026-07-30 13:47:09.670055454 +0000 UTC m=+2852.942519828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-maas-api-tls") pod "maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm" (UID: "71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485") : secret "maas-api-e2e-shared-a-f4dc0a-serving-cert" not found
Jul 30 13:47:09.170489 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.170469    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm\" (UID: \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 13:47:09.178835 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.178812    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8dt7\" (UniqueName: \"kubernetes.io/projected/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-kube-api-access-h8dt7\") pod \"maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm\" (UID: \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 13:47:09.673459 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.673433    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-maas-api-tls\") pod \"maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm\" (UID: \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 13:47:09.675773 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.675752    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-maas-api-tls\") pod \"maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm\" (UID: \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 13:47:09.676265 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.676244    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:47:09.774817 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.774739    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-data\") pod \"d4ad391a-c17d-4252-8cc4-666963b43c54\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") "
Jul 30 13:47:09.774953 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.774846    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-workload-socket\") pod \"d4ad391a-c17d-4252-8cc4-666963b43c54\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") "
Jul 30 13:47:09.774953 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.774886    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-podinfo\") pod \"d4ad391a-c17d-4252-8cc4-666963b43c54\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") "
Jul 30 13:47:09.774953 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.774909    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-workload-certs\") pod \"d4ad391a-c17d-4252-8cc4-666963b43c54\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") "
Jul 30 13:47:09.774953 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.774935    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4ad391a-c17d-4252-8cc4-666963b43c54-istiod-ca-cert\") pod \"d4ad391a-c17d-4252-8cc4-666963b43c54\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") "
Jul 30 13:47:09.775162 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.775041    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d4ad391a-c17d-4252-8cc4-666963b43c54" (UID: "d4ad391a-c17d-4252-8cc4-666963b43c54"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:47:09.775162 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.775085    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjzwh\" (UniqueName: \"kubernetes.io/projected/d4ad391a-c17d-4252-8cc4-666963b43c54-kube-api-access-kjzwh\") pod \"d4ad391a-c17d-4252-8cc4-666963b43c54\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") "
Jul 30 13:47:09.775162 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.775149    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-token\") pod \"d4ad391a-c17d-4252-8cc4-666963b43c54\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") "
Jul 30 13:47:09.775323 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.775177    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-envoy\") pod \"d4ad391a-c17d-4252-8cc4-666963b43c54\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") "
Jul 30 13:47:09.775323 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.775202    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-credential-socket\") pod \"d4ad391a-c17d-4252-8cc4-666963b43c54\" (UID: \"d4ad391a-c17d-4252-8cc4-666963b43c54\") "
Jul 30 13:47:09.775323 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.775227    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d4ad391a-c17d-4252-8cc4-666963b43c54" (UID: "d4ad391a-c17d-4252-8cc4-666963b43c54"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:47:09.775323 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.775303    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4ad391a-c17d-4252-8cc4-666963b43c54-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d4ad391a-c17d-4252-8cc4-666963b43c54" (UID: "d4ad391a-c17d-4252-8cc4-666963b43c54"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 13:47:09.775538 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.775523    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d4ad391a-c17d-4252-8cc4-666963b43c54" (UID: "d4ad391a-c17d-4252-8cc4-666963b43c54"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:47:09.775664 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.775645    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-credential-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:47:09.775753 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.775667    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-workload-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:47:09.775753 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.775681    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-workload-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:47:09.775753 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.775695    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4ad391a-c17d-4252-8cc4-666963b43c54-istiod-ca-cert\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:47:09.775753 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.775702    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-data" (OuterVolumeSpecName: "istio-data") pod "d4ad391a-c17d-4252-8cc4-666963b43c54" (UID: "d4ad391a-c17d-4252-8cc4-666963b43c54"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:47:09.777204 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.777182    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d4ad391a-c17d-4252-8cc4-666963b43c54" (UID: "d4ad391a-c17d-4252-8cc4-666963b43c54"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 13:47:09.777489 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.777469    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d4ad391a-c17d-4252-8cc4-666963b43c54" (UID: "d4ad391a-c17d-4252-8cc4-666963b43c54"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:47:09.777809 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.777784    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-token" (OuterVolumeSpecName: "istio-token") pod "d4ad391a-c17d-4252-8cc4-666963b43c54" (UID: "d4ad391a-c17d-4252-8cc4-666963b43c54"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:47:09.777903 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.777840    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4ad391a-c17d-4252-8cc4-666963b43c54-kube-api-access-kjzwh" (OuterVolumeSpecName: "kube-api-access-kjzwh") pod "d4ad391a-c17d-4252-8cc4-666963b43c54" (UID: "d4ad391a-c17d-4252-8cc4-666963b43c54"). InnerVolumeSpecName "kube-api-access-kjzwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:47:09.876565 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.876523    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjzwh\" (UniqueName: \"kubernetes.io/projected/d4ad391a-c17d-4252-8cc4-666963b43c54-kube-api-access-kjzwh\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:47:09.876565 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.876555    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-token\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:47:09.876565 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.876565    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-envoy\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:47:09.876565 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.876573    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-data\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:47:09.876565 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.876580    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4ad391a-c17d-4252-8cc4-666963b43c54-istio-podinfo\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:47:09.957179 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:09.957148    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 13:47:10.009918 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:10.009888    2557 generic.go:356] "Generic (PLEG): container finished" podID="d4ad391a-c17d-4252-8cc4-666963b43c54" containerID="5b77a988dafd79592e99bc463b1dc8576fd40cfa46ca4f07496f0af5d782a3d3" exitCode=0
Jul 30 13:47:10.010065 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:10.009952    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"
Jul 30 13:47:10.010065 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:10.009973    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl" event={"ID":"d4ad391a-c17d-4252-8cc4-666963b43c54","Type":"ContainerDied","Data":"5b77a988dafd79592e99bc463b1dc8576fd40cfa46ca4f07496f0af5d782a3d3"}
Jul 30 13:47:10.010192 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:10.010069    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl" event={"ID":"d4ad391a-c17d-4252-8cc4-666963b43c54","Type":"ContainerDied","Data":"e0f6c7fc5ea406d7c8ca3024c606bca26b83f51a71211920d5e243fa0d728ac9"}
Jul 30 13:47:10.010192 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:10.010137    2557 scope.go:117] "RemoveContainer" containerID="5b77a988dafd79592e99bc463b1dc8576fd40cfa46ca4f07496f0af5d782a3d3"
Jul 30 13:47:10.022336 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:10.022233    2557 scope.go:117] "RemoveContainer" containerID="d10863e6805b3d5fda4351f4f0a84a166266da1700b30d620447dc32c5265462"
Jul 30 13:47:10.032932 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:10.032885    2557 scope.go:117] "RemoveContainer" containerID="5b77a988dafd79592e99bc463b1dc8576fd40cfa46ca4f07496f0af5d782a3d3"
Jul 30 13:47:10.033200 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:47:10.033182    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b77a988dafd79592e99bc463b1dc8576fd40cfa46ca4f07496f0af5d782a3d3\": container with ID starting with 5b77a988dafd79592e99bc463b1dc8576fd40cfa46ca4f07496f0af5d782a3d3 not found: ID does not exist" containerID="5b77a988dafd79592e99bc463b1dc8576fd40cfa46ca4f07496f0af5d782a3d3"
Jul 30 13:47:10.033278 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:10.033213    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b77a988dafd79592e99bc463b1dc8576fd40cfa46ca4f07496f0af5d782a3d3"} err="failed to get container status \"5b77a988dafd79592e99bc463b1dc8576fd40cfa46ca4f07496f0af5d782a3d3\": rpc error: code = NotFound desc = could not find container \"5b77a988dafd79592e99bc463b1dc8576fd40cfa46ca4f07496f0af5d782a3d3\": container with ID starting with 5b77a988dafd79592e99bc463b1dc8576fd40cfa46ca4f07496f0af5d782a3d3 not found: ID does not exist"
Jul 30 13:47:10.033278 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:10.033238    2557 scope.go:117] "RemoveContainer" containerID="d10863e6805b3d5fda4351f4f0a84a166266da1700b30d620447dc32c5265462"
Jul 30 13:47:10.033688 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:47:10.033628    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d10863e6805b3d5fda4351f4f0a84a166266da1700b30d620447dc32c5265462\": container with ID starting with d10863e6805b3d5fda4351f4f0a84a166266da1700b30d620447dc32c5265462 not found: ID does not exist" containerID="d10863e6805b3d5fda4351f4f0a84a166266da1700b30d620447dc32c5265462"
Jul 30 13:47:10.033688 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:10.033664    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d10863e6805b3d5fda4351f4f0a84a166266da1700b30d620447dc32c5265462"} err="failed to get container status \"d10863e6805b3d5fda4351f4f0a84a166266da1700b30d620447dc32c5265462\": rpc error: code = NotFound desc = could not find container \"d10863e6805b3d5fda4351f4f0a84a166266da1700b30d620447dc32c5265462\": container with ID starting with d10863e6805b3d5fda4351f4f0a84a166266da1700b30d620447dc32c5265462 not found: ID does not exist"
Jul 30 13:47:10.037183 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:10.037158    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"]
Jul 30 13:47:10.041906 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:10.041883    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-5fdbfb-openshift-default-78bd958f8d-rfqbl"]
Jul 30 13:47:10.099436 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:10.099158    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"]
Jul 30 13:47:11.016523 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:11.016486    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm" event={"ID":"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485","Type":"ContainerStarted","Data":"be5b2307e9b39b4eecee57358c772fde149a5a7fbc3cee8c436089a8f86187be"}
Jul 30 13:47:11.184095 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:11.184057    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4ad391a-c17d-4252-8cc4-666963b43c54" path="/var/lib/kubelet/pods/d4ad391a-c17d-4252-8cc4-666963b43c54/volumes"
Jul 30 13:47:12.022840 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:12.022801    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm" event={"ID":"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485","Type":"ContainerStarted","Data":"77f1f91de5847054ef1cc870a78e4c0affcad5ad948a113b61ac9243af67a8f9"}
Jul 30 13:47:12.023037 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:12.022935    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 13:47:12.042220 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:12.042169    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm" podStartSLOduration=1.6328460040000001 podStartE2EDuration="3.042153615s" totalImagesPullingTime="1.409307611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:47:12.039908152 +0000 UTC m=+2855.312372567" watchObservedRunningTime="2026-07-30 13:47:12.042153615 +0000 UTC m=+2855.314618037"
Jul 30 13:47:14.528397 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:14.528362    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:47:18.033308 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:18.033278    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 13:47:22.550017 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:22.549521    2557 patch_prober.go:28] interesting pod/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": read tcp 10.134.0.2:50570->10.134.0.52:15021: read: connection reset by peer" start-of-body=
Jul 30 13:47:22.550017 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:22.549588    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc" podUID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": read tcp 10.134.0.2:50570->10.134.0.52:15021: read: connection reset by peer"
Jul 30 13:47:23.078057 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:23.078023    2557 generic.go:356] "Generic (PLEG): container finished" podID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerID="cba115ce1db2a6771bbc2a766e98a07fcd67fcea6ccf46214c92ebdb56115a34" exitCode=0
Jul 30 13:47:23.078226 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:23.078098    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc" event={"ID":"d1a35c54-1b67-4918-bdea-5b1a270b0a34","Type":"ContainerDied","Data":"cba115ce1db2a6771bbc2a766e98a07fcd67fcea6ccf46214c92ebdb56115a34"}
Jul 30 13:47:23.078624 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:23.078584    2557 scope.go:117] "RemoveContainer" containerID="cba115ce1db2a6771bbc2a766e98a07fcd67fcea6ccf46214c92ebdb56115a34"
Jul 30 13:47:23.079730 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:23.079702    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:47:23.079798 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:23.079773    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:47:23.079838 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:23.079805    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:47:24.096861 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:24.096815    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc" event={"ID":"d1a35c54-1b67-4918-bdea-5b1a270b0a34","Type":"ContainerStarted","Data":"1f8a9c551da3e21bc665a4018ef832cddbdaa8aae5106268c76c296aff10d047"}
Jul 30 13:47:24.604116 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:24.604076    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:24.604372 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:24.604280    2557 patch_prober.go:28] interesting pod/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 13:47:24.604372 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:24.604343    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc" podUID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 13:47:25.604016 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:25.603978    2557 patch_prober.go:28] interesting pod/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 13:47:25.604435 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:25.604035    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc" podUID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 13:47:26.604100 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:26.604063    2557 patch_prober.go:28] interesting pod/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 13:47:26.604582 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:26.604125    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc" podUID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 13:47:27.603761 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:27.603724    2557 patch_prober.go:28] interesting pod/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 13:47:27.603955 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:27.603782    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc" podUID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 13:47:28.607186 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:28.607154    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:28.607624 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:28.607428    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:28.608093 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:28.608079    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 13:47:33.598039 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.598007    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"]
Jul 30 13:47:33.598435 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.598346    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ad391a-c17d-4252-8cc4-666963b43c54" containerName="istio-proxy"
Jul 30 13:47:33.598435 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.598358    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ad391a-c17d-4252-8cc4-666963b43c54" containerName="istio-proxy"
Jul 30 13:47:33.598435 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.598433    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4ad391a-c17d-4252-8cc4-666963b43c54" containerName="istio-proxy"
Jul 30 13:47:33.601880 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.601859    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.605387 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.605365    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-e945a1-openshift-default-dockercfg-xglsx\""
Jul 30 13:47:33.617703 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.617682    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"]
Jul 30 13:47:33.691336 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.691294    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-envoy\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.691543 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.691342    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-data\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.691543 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.691427    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6258642-5130-4e8b-ad53-2bdb682f8b29-istiod-ca-cert\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.691543 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.691469    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-podinfo\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.691543 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.691493    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fvfw\" (UniqueName: \"kubernetes.io/projected/e6258642-5130-4e8b-ad53-2bdb682f8b29-kube-api-access-6fvfw\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.691716 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.691614    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-credential-socket\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.691716 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.691661    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-workload-socket\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.691716 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.691711    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-token\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.691834 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.691812    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-workload-certs\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.792775 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.792744    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-token\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.792990 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.792804    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-workload-certs\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.792990 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.792831    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-envoy\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.792990 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.792848    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-data\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.792990 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.792868    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6258642-5130-4e8b-ad53-2bdb682f8b29-istiod-ca-cert\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.792990 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.792890    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-podinfo\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.793288 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.793016    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fvfw\" (UniqueName: \"kubernetes.io/projected/e6258642-5130-4e8b-ad53-2bdb682f8b29-kube-api-access-6fvfw\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.793288 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.793082    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-credential-socket\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.793288 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.793257    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-workload-socket\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.793454 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.793302    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-data\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.793454 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.793422    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-workload-certs\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.793454 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.793433    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-credential-socket\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.793689 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.793647    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-workload-socket\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.793825 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.793808    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6258642-5130-4e8b-ad53-2bdb682f8b29-istiod-ca-cert\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.795278 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.795260    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-envoy\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.795409 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.795393    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-podinfo\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.801741 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.801720    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-token\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.801937 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.801917    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fvfw\" (UniqueName: \"kubernetes.io/projected/e6258642-5130-4e8b-ad53-2bdb682f8b29-kube-api-access-6fvfw\") pod \"e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") " pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:33.912475 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:33.912447    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:34.045965 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:34.045938    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"]
Jul 30 13:47:34.047462 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:47:34.047427    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6258642_5130_4e8b_ad53_2bdb682f8b29.slice/crio-d8862b1de37874b59ee8bb18a39aa8a7a623bac5e5730e1ad9a4f3811510908e WatchSource:0}: Error finding container d8862b1de37874b59ee8bb18a39aa8a7a623bac5e5730e1ad9a4f3811510908e: Status 404 returned error can't find the container with id d8862b1de37874b59ee8bb18a39aa8a7a623bac5e5730e1ad9a4f3811510908e
Jul 30 13:47:34.049533 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:34.049502    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:47:34.049617 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:34.049560    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:47:34.049617 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:34.049587    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:47:34.143614 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:34.143567    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z" event={"ID":"e6258642-5130-4e8b-ad53-2bdb682f8b29","Type":"ContainerStarted","Data":"0b6418807394f4dcf46f9175a3e944525aca761d1cb9ba972c39e7867fc9458a"}
Jul 30 13:47:34.143779 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:34.143633    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z" event={"ID":"e6258642-5130-4e8b-ad53-2bdb682f8b29","Type":"ContainerStarted","Data":"d8862b1de37874b59ee8bb18a39aa8a7a623bac5e5730e1ad9a4f3811510908e"}
Jul 30 13:47:34.170448 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:34.170343    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z" podStartSLOduration=1.170328008 podStartE2EDuration="1.170328008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:47:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:47:34.169803508 +0000 UTC m=+2877.442267925" watchObservedRunningTime="2026-07-30 13:47:34.170328008 +0000 UTC m=+2877.442792382"
Jul 30 13:47:34.913165 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:34.913117    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:35.917754 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:35.917718    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:36.153212 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:36.153181    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:36.154246 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:36.154229    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:44.132335 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:44.132302    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:47:51.953849 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:51.952626    2557 patch_prober.go:28] interesting pod/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:56140->10.134.0.54:15021: read: connection reset by peer" start-of-body=
Jul 30 13:47:51.953849 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:51.952697    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z" podUID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:56140->10.134.0.54:15021: read: connection reset by peer"
Jul 30 13:47:52.233957 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:52.233866    2557 generic.go:356] "Generic (PLEG): container finished" podID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerID="0b6418807394f4dcf46f9175a3e944525aca761d1cb9ba972c39e7867fc9458a" exitCode=0
Jul 30 13:47:52.233957 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:52.233943    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z" event={"ID":"e6258642-5130-4e8b-ad53-2bdb682f8b29","Type":"ContainerDied","Data":"0b6418807394f4dcf46f9175a3e944525aca761d1cb9ba972c39e7867fc9458a"}
Jul 30 13:47:52.234354 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:52.234340    2557 scope.go:117] "RemoveContainer" containerID="0b6418807394f4dcf46f9175a3e944525aca761d1cb9ba972c39e7867fc9458a"
Jul 30 13:47:52.235384 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:52.235357    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:47:52.235489 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:52.235423    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:47:52.235489 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:52.235454    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:47:53.240206 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:53.240170    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z" event={"ID":"e6258642-5130-4e8b-ad53-2bdb682f8b29","Type":"ContainerStarted","Data":"fc4f7060e19687ab161f0df182fff36d7bc7731466a07f6471826f5a0539ab12"}
Jul 30 13:47:53.913528 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:53.913493    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:53.913781 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:53.913757    2557 patch_prober.go:28] interesting pod/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 30 13:47:53.913840 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:53.913821    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z" podUID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 30 13:47:54.913510 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:54.913470    2557 patch_prober.go:28] interesting pod/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 30 13:47:54.914120 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:54.913533    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z" podUID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 30 13:47:55.912930 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:55.912893    2557 patch_prober.go:28] interesting pod/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 30 13:47:55.913122 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:55.912947    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z" podUID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 30 13:47:56.913774 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:56.913733    2557 patch_prober.go:28] interesting pod/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 30 13:47:56.914273 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:56.913793    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z" podUID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 30 13:47:57.916436 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:57.916406    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:57.916841 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:57.916629    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:47:57.917319 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:47:57.917301    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 13:48:11.437833 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.437796    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"]
Jul 30 13:48:11.438368 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.438348    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ad391a-c17d-4252-8cc4-666963b43c54" containerName="istio-proxy"
Jul 30 13:48:11.438444 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.438371    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ad391a-c17d-4252-8cc4-666963b43c54" containerName="istio-proxy"
Jul 30 13:48:11.438503 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.438489    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4ad391a-c17d-4252-8cc4-666963b43c54" containerName="istio-proxy"
Jul 30 13:48:11.441769 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.441749    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.446274 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.446245    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f4dc0a\"/\"kube-root-ca.crt\""
Jul 30 13:48:11.446399 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.446269    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f4dc0a\"/\"openshift-service-ca.crt\""
Jul 30 13:48:11.446399 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.446327    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f4dc0a\"/\"default-dockercfg-6r788\""
Jul 30 13:48:11.446399 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.446327    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f4dc0a\"/\"auth-test-model-f4dc0a-kserve-self-signed-certs\""
Jul 30 13:48:11.456111 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.456091    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"]
Jul 30 13:48:11.543783 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.543751    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-tmp-dir\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.543963 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.543789    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6756r\" (UniqueName: \"kubernetes.io/projected/eed1218b-d96d-4d97-82ea-028bb4d67080-kube-api-access-6756r\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.543963 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.543849    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eed1218b-d96d-4d97-82ea-028bb4d67080-tls-certs\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.544054 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.543975    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-model-cache\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.544054 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.544025    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-dshm\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.544156 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.544099    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-home\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.645495 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.645463    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-dshm\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.645677 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.645512    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-home\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.645677 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.645565    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-tmp-dir\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.645677 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.645590    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6756r\" (UniqueName: \"kubernetes.io/projected/eed1218b-d96d-4d97-82ea-028bb4d67080-kube-api-access-6756r\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.645677 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.645651    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eed1218b-d96d-4d97-82ea-028bb4d67080-tls-certs\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.645890 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.645713    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-model-cache\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.646027 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.646001    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-home\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.646116 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.646062    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-tmp-dir\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.646116 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.646067    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-model-cache\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.647729 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.647702    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-dshm\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.648125 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.648104    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eed1218b-d96d-4d97-82ea-028bb4d67080-tls-certs\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.654775 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.654748    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6756r\" (UniqueName: \"kubernetes.io/projected/eed1218b-d96d-4d97-82ea-028bb4d67080-kube-api-access-6756r\") pod \"auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") " pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.751293 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.751193    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:11.882789 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:11.882761    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"]
Jul 30 13:48:11.884786 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:48:11.884757    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeed1218b_d96d_4d97_82ea_028bb4d67080.slice/crio-9201deee34acb4c5be231a369439bff6fb2b3d38b03d9351fd59be748a40ba46 WatchSource:0}: Error finding container 9201deee34acb4c5be231a369439bff6fb2b3d38b03d9351fd59be748a40ba46: Status 404 returned error can't find the container with id 9201deee34acb4c5be231a369439bff6fb2b3d38b03d9351fd59be748a40ba46
Jul 30 13:48:12.331285 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:12.331247    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp" event={"ID":"eed1218b-d96d-4d97-82ea-028bb4d67080","Type":"ContainerStarted","Data":"28e2a8c697191632bf135f37012a8ca6c167c54768cfd231636fac5868d8024f"}
Jul 30 13:48:12.331285 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:12.331289    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp" event={"ID":"eed1218b-d96d-4d97-82ea-028bb4d67080","Type":"ContainerStarted","Data":"9201deee34acb4c5be231a369439bff6fb2b3d38b03d9351fd59be748a40ba46"}
Jul 30 13:48:12.353535 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:12.353483    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp" podStartSLOduration=1.353466377 podStartE2EDuration="1.353466377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:48:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:48:12.350266343 +0000 UTC m=+2915.622730744" watchObservedRunningTime="2026-07-30 13:48:12.353466377 +0000 UTC m=+2915.625930774"
Jul 30 13:48:15.235771 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:15.235733    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:48:21.752158 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:21.752102    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:21.752158 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:21.752166    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:21.764862 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:21.764834    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:22.390463 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:22.390437    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:48:33.966251 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:33.966221    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"]
Jul 30 13:48:33.969653 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:33.969633    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:33.972723 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:33.972696    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e945a1\"/\"auth-test-model-e945a1-kserve-self-signed-certs\""
Jul 30 13:48:33.974026 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:33.974008    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e945a1\"/\"kube-root-ca.crt\""
Jul 30 13:48:33.974134 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:33.974019    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e945a1\"/\"openshift-service-ca.crt\""
Jul 30 13:48:33.974134 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:33.974108    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e945a1\"/\"default-dockercfg-jbtzb\""
Jul 30 13:48:33.980445 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:33.980421    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"]
Jul 30 13:48:34.048835 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.048801    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-tmp-dir\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.048835 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.048839    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-home\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.049060 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.048895    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-model-cache\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.049060 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.048937    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0365baef-e369-46e6-a0e6-463bd149cefb-tls-certs\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.049060 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.048971    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7drkh\" (UniqueName: \"kubernetes.io/projected/0365baef-e369-46e6-a0e6-463bd149cefb-kube-api-access-7drkh\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.049060 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.049041    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-dshm\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.149900 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.149871    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-dshm\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.150077 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.149918    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-tmp-dir\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.150077 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.150036    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-home\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.150153 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.150092    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-model-cache\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.150153 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.150132    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0365baef-e369-46e6-a0e6-463bd149cefb-tls-certs\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.150232 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.150166    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7drkh\" (UniqueName: \"kubernetes.io/projected/0365baef-e369-46e6-a0e6-463bd149cefb-kube-api-access-7drkh\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.150380 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.150361    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-tmp-dir\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.150436 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.150411    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-home\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.150483 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.150455    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-model-cache\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.152130 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.152108    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-dshm\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.152548 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.152533    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0365baef-e369-46e6-a0e6-463bd149cefb-tls-certs\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.159833 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.159814    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7drkh\" (UniqueName: \"kubernetes.io/projected/0365baef-e369-46e6-a0e6-463bd149cefb-kube-api-access-7drkh\") pod \"auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") " pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.279370 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.279274    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:34.413851 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.413825    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"]
Jul 30 13:48:34.415713 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:48:34.415686    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0365baef_e369_46e6_a0e6_463bd149cefb.slice/crio-7142ed26a5c1bdc3b551fc87e230af23ec57e47616ce28e3f705307afbc94149 WatchSource:0}: Error finding container 7142ed26a5c1bdc3b551fc87e230af23ec57e47616ce28e3f705307afbc94149: Status 404 returned error can't find the container with id 7142ed26a5c1bdc3b551fc87e230af23ec57e47616ce28e3f705307afbc94149
Jul 30 13:48:34.436725 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:34.436687    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn" event={"ID":"0365baef-e369-46e6-a0e6-463bd149cefb","Type":"ContainerStarted","Data":"7142ed26a5c1bdc3b551fc87e230af23ec57e47616ce28e3f705307afbc94149"}
Jul 30 13:48:35.442388 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:35.442353    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn" event={"ID":"0365baef-e369-46e6-a0e6-463bd149cefb","Type":"ContainerStarted","Data":"50eb109ca41f7467728eb66d9417725edd1b28bf110a595089b829168c0796fe"}
Jul 30 13:48:35.465191 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:35.465140    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn" podStartSLOduration=2.465125556 podStartE2EDuration="2.465125556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:48:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:48:35.46381997 +0000 UTC m=+2938.736284368" watchObservedRunningTime="2026-07-30 13:48:35.465125556 +0000 UTC m=+2938.737589953"
Jul 30 13:48:37.645336 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:37.645304    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:48:44.279491 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:44.279445    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:44.279491 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:44.279503    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:44.291924 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:44.291896    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:48:44.497850 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:48:44.497823    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:49:02.644037 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:49:02.643958    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:49:28.345940 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:49:28.345904    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:49:46.437270 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:49:46.437235    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:50:14.031379 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:50:14.031328    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:50:27.332829 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:50:27.332791    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:50:38.842787 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:50:38.842697    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:51:00.531240 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:51:00.531203    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:51:18.040832 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:51:18.040800    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:51:37.528633 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:51:37.528586    2557 scope.go:117] "RemoveContainer" containerID="6a9a3b2319d85b09d59db16ee85802d4364db4526238b15cfb1372c6bba92dee"
Jul 30 13:51:45.833590 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:51:45.833555    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:52:00.344024 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:52:00.343941    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:52:11.447949 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:52:11.447914    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:52:31.344577 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:52:31.344539    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:52:49.044135 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:52:49.044098    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:53:15.735990 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:15.735954    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:53:20.954054 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:20.954013    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"]
Jul 30 13:53:22.017241 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:22.017204    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"]
Jul 30 13:53:33.336193 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:33.336151    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:53:35.972832 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:35.972771    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp" podUID="eed1218b-d96d-4d97-82ea-028bb4d67080" containerName="main" containerID="cri-o://28e2a8c697191632bf135f37012a8ca6c167c54768cfd231636fac5868d8024f" gracePeriod=45
Jul 30 13:53:36.229833 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.229768    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:53:36.259568 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.259543    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-home\") pod \"eed1218b-d96d-4d97-82ea-028bb4d67080\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") "
Jul 30 13:53:36.259765 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.259622    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-tmp-dir\") pod \"eed1218b-d96d-4d97-82ea-028bb4d67080\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") "
Jul 30 13:53:36.259765 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.259656    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eed1218b-d96d-4d97-82ea-028bb4d67080-tls-certs\") pod \"eed1218b-d96d-4d97-82ea-028bb4d67080\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") "
Jul 30 13:53:36.259765 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.259683    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-dshm\") pod \"eed1218b-d96d-4d97-82ea-028bb4d67080\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") "
Jul 30 13:53:36.259765 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.259710    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-model-cache\") pod \"eed1218b-d96d-4d97-82ea-028bb4d67080\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") "
Jul 30 13:53:36.259968 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.259806    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6756r\" (UniqueName: \"kubernetes.io/projected/eed1218b-d96d-4d97-82ea-028bb4d67080-kube-api-access-6756r\") pod \"eed1218b-d96d-4d97-82ea-028bb4d67080\" (UID: \"eed1218b-d96d-4d97-82ea-028bb4d67080\") "
Jul 30 13:53:36.259968 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.259863    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-home" (OuterVolumeSpecName: "home") pod "eed1218b-d96d-4d97-82ea-028bb4d67080" (UID: "eed1218b-d96d-4d97-82ea-028bb4d67080"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:53:36.260050 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.259965    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eed1218b-d96d-4d97-82ea-028bb4d67080" (UID: "eed1218b-d96d-4d97-82ea-028bb4d67080"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:53:36.260050 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.259975    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-model-cache" (OuterVolumeSpecName: "model-cache") pod "eed1218b-d96d-4d97-82ea-028bb4d67080" (UID: "eed1218b-d96d-4d97-82ea-028bb4d67080"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:53:36.260422 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.260388    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-home\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:53:36.260422 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.260416    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-tmp-dir\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:53:36.260585 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.260428    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-model-cache\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:53:36.262089 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.262055    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eed1218b-d96d-4d97-82ea-028bb4d67080-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eed1218b-d96d-4d97-82ea-028bb4d67080" (UID: "eed1218b-d96d-4d97-82ea-028bb4d67080"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 13:53:36.262331 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.262304    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-dshm" (OuterVolumeSpecName: "dshm") pod "eed1218b-d96d-4d97-82ea-028bb4d67080" (UID: "eed1218b-d96d-4d97-82ea-028bb4d67080"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:53:36.262331 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.262319    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eed1218b-d96d-4d97-82ea-028bb4d67080-kube-api-access-6756r" (OuterVolumeSpecName: "kube-api-access-6756r") pod "eed1218b-d96d-4d97-82ea-028bb4d67080" (UID: "eed1218b-d96d-4d97-82ea-028bb4d67080"). InnerVolumeSpecName "kube-api-access-6756r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:53:36.361063 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.361023    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eed1218b-d96d-4d97-82ea-028bb4d67080-tls-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:53:36.361063 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.361055    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eed1218b-d96d-4d97-82ea-028bb4d67080-dshm\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:53:36.361063 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.361066    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6756r\" (UniqueName: \"kubernetes.io/projected/eed1218b-d96d-4d97-82ea-028bb4d67080-kube-api-access-6756r\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:53:36.882186 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.882155    2557 generic.go:356] "Generic (PLEG): container finished" podID="eed1218b-d96d-4d97-82ea-028bb4d67080" containerID="28e2a8c697191632bf135f37012a8ca6c167c54768cfd231636fac5868d8024f" exitCode=0
Jul 30 13:53:36.882342 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.882197    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"
Jul 30 13:53:36.882342 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.882233    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp" event={"ID":"eed1218b-d96d-4d97-82ea-028bb4d67080","Type":"ContainerDied","Data":"28e2a8c697191632bf135f37012a8ca6c167c54768cfd231636fac5868d8024f"}
Jul 30 13:53:36.882342 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.882274    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp" event={"ID":"eed1218b-d96d-4d97-82ea-028bb4d67080","Type":"ContainerDied","Data":"9201deee34acb4c5be231a369439bff6fb2b3d38b03d9351fd59be748a40ba46"}
Jul 30 13:53:36.882342 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.882293    2557 scope.go:117] "RemoveContainer" containerID="28e2a8c697191632bf135f37012a8ca6c167c54768cfd231636fac5868d8024f"
Jul 30 13:53:36.891945 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.891929    2557 scope.go:117] "RemoveContainer" containerID="28e2a8c697191632bf135f37012a8ca6c167c54768cfd231636fac5868d8024f"
Jul 30 13:53:36.892201 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:53:36.892181    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28e2a8c697191632bf135f37012a8ca6c167c54768cfd231636fac5868d8024f\": container with ID starting with 28e2a8c697191632bf135f37012a8ca6c167c54768cfd231636fac5868d8024f not found: ID does not exist" containerID="28e2a8c697191632bf135f37012a8ca6c167c54768cfd231636fac5868d8024f"
Jul 30 13:53:36.892247 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.892209    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28e2a8c697191632bf135f37012a8ca6c167c54768cfd231636fac5868d8024f"} err="failed to get container status \"28e2a8c697191632bf135f37012a8ca6c167c54768cfd231636fac5868d8024f\": rpc error: code = NotFound desc = could not find container \"28e2a8c697191632bf135f37012a8ca6c167c54768cfd231636fac5868d8024f\": container with ID starting with 28e2a8c697191632bf135f37012a8ca6c167c54768cfd231636fac5868d8024f not found: ID does not exist"
Jul 30 13:53:36.906084 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.906062    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"]
Jul 30 13:53:36.911867 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:36.911848    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/auth-test-model-f4dc0a-kserve-6cf8685fbc-4b8vp"]
Jul 30 13:53:37.036848 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.036809    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn" podUID="0365baef-e369-46e6-a0e6-463bd149cefb" containerName="main" containerID="cri-o://50eb109ca41f7467728eb66d9417725edd1b28bf110a595089b829168c0796fe" gracePeriod=45
Jul 30 13:53:37.190488 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.190451    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eed1218b-d96d-4d97-82ea-028bb4d67080" path="/var/lib/kubelet/pods/eed1218b-d96d-4d97-82ea-028bb4d67080/volumes"
Jul 30 13:53:37.300249 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.300222    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:53:37.370570 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.370533    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0365baef-e369-46e6-a0e6-463bd149cefb-tls-certs\") pod \"0365baef-e369-46e6-a0e6-463bd149cefb\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") "
Jul 30 13:53:37.370796 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.370615    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-dshm\") pod \"0365baef-e369-46e6-a0e6-463bd149cefb\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") "
Jul 30 13:53:37.370796 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.370673    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-model-cache\") pod \"0365baef-e369-46e6-a0e6-463bd149cefb\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") "
Jul 30 13:53:37.370796 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.370706    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-tmp-dir\") pod \"0365baef-e369-46e6-a0e6-463bd149cefb\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") "
Jul 30 13:53:37.370796 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.370733    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-home\") pod \"0365baef-e369-46e6-a0e6-463bd149cefb\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") "
Jul 30 13:53:37.370796 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.370759    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7drkh\" (UniqueName: \"kubernetes.io/projected/0365baef-e369-46e6-a0e6-463bd149cefb-kube-api-access-7drkh\") pod \"0365baef-e369-46e6-a0e6-463bd149cefb\" (UID: \"0365baef-e369-46e6-a0e6-463bd149cefb\") "
Jul 30 13:53:37.371049 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.370965    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0365baef-e369-46e6-a0e6-463bd149cefb" (UID: "0365baef-e369-46e6-a0e6-463bd149cefb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:53:37.371049 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.370977    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-home" (OuterVolumeSpecName: "home") pod "0365baef-e369-46e6-a0e6-463bd149cefb" (UID: "0365baef-e369-46e6-a0e6-463bd149cefb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:53:37.371049 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.370989    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-model-cache" (OuterVolumeSpecName: "model-cache") pod "0365baef-e369-46e6-a0e6-463bd149cefb" (UID: "0365baef-e369-46e6-a0e6-463bd149cefb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:53:37.371172 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.371122    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-model-cache\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:53:37.371172 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.371139    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-tmp-dir\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:53:37.371172 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.371153    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-home\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:53:37.372858 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.372829    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-dshm" (OuterVolumeSpecName: "dshm") pod "0365baef-e369-46e6-a0e6-463bd149cefb" (UID: "0365baef-e369-46e6-a0e6-463bd149cefb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:53:37.372985 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.372879    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0365baef-e369-46e6-a0e6-463bd149cefb-kube-api-access-7drkh" (OuterVolumeSpecName: "kube-api-access-7drkh") pod "0365baef-e369-46e6-a0e6-463bd149cefb" (UID: "0365baef-e369-46e6-a0e6-463bd149cefb"). InnerVolumeSpecName "kube-api-access-7drkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:53:37.372985 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.372953    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0365baef-e369-46e6-a0e6-463bd149cefb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0365baef-e369-46e6-a0e6-463bd149cefb" (UID: "0365baef-e369-46e6-a0e6-463bd149cefb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 13:53:37.471711 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.471674    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0365baef-e369-46e6-a0e6-463bd149cefb-dshm\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:53:37.471711 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.471706    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7drkh\" (UniqueName: \"kubernetes.io/projected/0365baef-e369-46e6-a0e6-463bd149cefb-kube-api-access-7drkh\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:53:37.471711 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.471718    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0365baef-e369-46e6-a0e6-463bd149cefb-tls-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:53:37.887928 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.887845    2557 generic.go:356] "Generic (PLEG): container finished" podID="0365baef-e369-46e6-a0e6-463bd149cefb" containerID="50eb109ca41f7467728eb66d9417725edd1b28bf110a595089b829168c0796fe" exitCode=0
Jul 30 13:53:37.887928 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.887891    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"
Jul 30 13:53:37.888126 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.887935    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn" event={"ID":"0365baef-e369-46e6-a0e6-463bd149cefb","Type":"ContainerDied","Data":"50eb109ca41f7467728eb66d9417725edd1b28bf110a595089b829168c0796fe"}
Jul 30 13:53:37.888126 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.887982    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn" event={"ID":"0365baef-e369-46e6-a0e6-463bd149cefb","Type":"ContainerDied","Data":"7142ed26a5c1bdc3b551fc87e230af23ec57e47616ce28e3f705307afbc94149"}
Jul 30 13:53:37.888126 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.888007    2557 scope.go:117] "RemoveContainer" containerID="50eb109ca41f7467728eb66d9417725edd1b28bf110a595089b829168c0796fe"
Jul 30 13:53:37.896999 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.896981    2557 scope.go:117] "RemoveContainer" containerID="50eb109ca41f7467728eb66d9417725edd1b28bf110a595089b829168c0796fe"
Jul 30 13:53:37.897265 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:53:37.897243    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50eb109ca41f7467728eb66d9417725edd1b28bf110a595089b829168c0796fe\": container with ID starting with 50eb109ca41f7467728eb66d9417725edd1b28bf110a595089b829168c0796fe not found: ID does not exist" containerID="50eb109ca41f7467728eb66d9417725edd1b28bf110a595089b829168c0796fe"
Jul 30 13:53:37.897312 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.897276    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50eb109ca41f7467728eb66d9417725edd1b28bf110a595089b829168c0796fe"} err="failed to get container status \"50eb109ca41f7467728eb66d9417725edd1b28bf110a595089b829168c0796fe\": rpc error: code = NotFound desc = could not find container \"50eb109ca41f7467728eb66d9417725edd1b28bf110a595089b829168c0796fe\": container with ID starting with 50eb109ca41f7467728eb66d9417725edd1b28bf110a595089b829168c0796fe not found: ID does not exist"
Jul 30 13:53:37.932652 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.932625    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"]
Jul 30 13:53:37.942181 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:37.942161    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/auth-test-model-e945a1-kserve-7c8cb6cddc-6wzvn"]
Jul 30 13:53:39.184162 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:39.184129    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0365baef-e369-46e6-a0e6-463bd149cefb" path="/var/lib/kubelet/pods/0365baef-e369-46e6-a0e6-463bd149cefb/volumes"
Jul 30 13:53:54.464990 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.464956    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"]
Jul 30 13:53:54.465460 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.465310    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eed1218b-d96d-4d97-82ea-028bb4d67080" containerName="main"
Jul 30 13:53:54.465460 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.465321    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="eed1218b-d96d-4d97-82ea-028bb4d67080" containerName="main"
Jul 30 13:53:54.465460 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.465336    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0365baef-e369-46e6-a0e6-463bd149cefb" containerName="main"
Jul 30 13:53:54.465460 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.465342    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0365baef-e369-46e6-a0e6-463bd149cefb" containerName="main"
Jul 30 13:53:54.465460 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.465399    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="eed1218b-d96d-4d97-82ea-028bb4d67080" containerName="main"
Jul 30 13:53:54.465460 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.465408    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0365baef-e369-46e6-a0e6-463bd149cefb" containerName="main"
Jul 30 13:53:54.468432 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.468416    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.471521 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.471495    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f4dc0a\"/\"openshift-service-ca.crt\""
Jul 30 13:53:54.471650 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.471547    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f4dc0a\"/\"default-dockercfg-6r788\""
Jul 30 13:53:54.472980 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.472956    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f4dc0a\"/\"sub-test-model-f4dc0a-kserve-self-signed-certs\""
Jul 30 13:53:54.473095 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.472990    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f4dc0a\"/\"kube-root-ca.crt\""
Jul 30 13:53:54.478014 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.477995    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"]
Jul 30 13:53:54.531206 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.531170    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z8bq\" (UniqueName: \"kubernetes.io/projected/60936de5-432b-4426-9126-ff32e822e3bd-kube-api-access-7z8bq\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.531375 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.531226    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-model-cache\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.531375 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.531248    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-home\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.531375 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.531264    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-dshm\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.531375 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.531319    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60936de5-432b-4426-9126-ff32e822e3bd-tls-certs\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.531557 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.531420    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-tmp-dir\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.632682 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.632636    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60936de5-432b-4426-9126-ff32e822e3bd-tls-certs\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.632868 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.632785    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-tmp-dir\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.632868 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.632813    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z8bq\" (UniqueName: \"kubernetes.io/projected/60936de5-432b-4426-9126-ff32e822e3bd-kube-api-access-7z8bq\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.633007 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.632960    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-model-cache\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.633007 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.632997    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-home\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.633128 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.633018    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-dshm\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.633279 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.633257    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-tmp-dir\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.633353 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.633291    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-model-cache\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.633353 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.633332    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-home\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.635172 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.635151    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-dshm\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.635172 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.635167    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60936de5-432b-4426-9126-ff32e822e3bd-tls-certs\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.643772 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.643740    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z8bq\" (UniqueName: \"kubernetes.io/projected/60936de5-432b-4426-9126-ff32e822e3bd-kube-api-access-7z8bq\") pod \"sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") " pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.779320 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.779239    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:53:54.927623 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.927579    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"]
Jul 30 13:53:54.930094 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:53:54.930064    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60936de5_432b_4426_9126_ff32e822e3bd.slice/crio-a18fe0a15f433cae9518fb55f4695dab361a45b26a52f9c8b16b63614694b2f6 WatchSource:0}: Error finding container a18fe0a15f433cae9518fb55f4695dab361a45b26a52f9c8b16b63614694b2f6: Status 404 returned error can't find the container with id a18fe0a15f433cae9518fb55f4695dab361a45b26a52f9c8b16b63614694b2f6
Jul 30 13:53:54.968547 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:54.968522    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z" event={"ID":"60936de5-432b-4426-9126-ff32e822e3bd","Type":"ContainerStarted","Data":"a18fe0a15f433cae9518fb55f4695dab361a45b26a52f9c8b16b63614694b2f6"}
Jul 30 13:53:55.974341 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:55.974298    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z" event={"ID":"60936de5-432b-4426-9126-ff32e822e3bd","Type":"ContainerStarted","Data":"bb94e369a71fd143670ff5927d5e41863b0f262f14946616bbf39a1450c4022b"}
Jul 30 13:53:56.003732 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:56.003682    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z" podStartSLOduration=2.003666046 podStartE2EDuration="2.003666046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:53:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:53:56.000732798 +0000 UTC m=+3259.273197207" watchObservedRunningTime="2026-07-30 13:53:56.003666046 +0000 UTC m=+3259.276130443"
Jul 30 13:53:57.850925 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:53:57.850889    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:54:04.779869 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:04.779829    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:54:04.779869 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:04.779877    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:54:04.792576 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:04.792549    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:54:05.027672 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:05.027638    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:54:14.865667 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:14.865628    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"]
Jul 30 13:54:14.869181 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:14.869163    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:14.874123 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:14.874100    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e945a1\"/\"default-dockercfg-jbtzb\""
Jul 30 13:54:14.874123 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:14.874119    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e945a1\"/\"sub-test-model-e945a1-kserve-self-signed-certs\""
Jul 30 13:54:14.874287 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:14.874118    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e945a1\"/\"openshift-service-ca.crt\""
Jul 30 13:54:14.874634 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:14.874587    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e945a1\"/\"kube-root-ca.crt\""
Jul 30 13:54:14.881808 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:14.881783    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"]
Jul 30 13:54:14.922949 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:14.922916    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-tmp-dir\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:14.922949 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:14.922950    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-model-cache\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:14.923170 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:14.923051    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-home\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:14.923170 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:14.923084    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-dshm\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:14.923248 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:14.923164    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a992d0b5-0a4f-4795-8aea-9230e65f712e-tls-certs\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:14.923248 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:14.923189    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmcsk\" (UniqueName: \"kubernetes.io/projected/a992d0b5-0a4f-4795-8aea-9230e65f712e-kube-api-access-nmcsk\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:15.023778 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:15.023745    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-tmp-dir\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:15.023778 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:15.023784    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-model-cache\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:15.024041 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:15.023844    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-home\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:15.024041 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:15.023861    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-dshm\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:15.024041 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:15.023950    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a992d0b5-0a4f-4795-8aea-9230e65f712e-tls-certs\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:15.024041 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:15.023990    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmcsk\" (UniqueName: \"kubernetes.io/projected/a992d0b5-0a4f-4795-8aea-9230e65f712e-kube-api-access-nmcsk\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:15.024247 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:15.024186    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-tmp-dir\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:15.024247 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:15.024234    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-model-cache\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:15.024353 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:15.024259    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-home\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:15.026163 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:15.026146    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-dshm\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:15.026418 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:15.026400    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a992d0b5-0a4f-4795-8aea-9230e65f712e-tls-certs\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:15.036036 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:15.036012    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmcsk\" (UniqueName: \"kubernetes.io/projected/a992d0b5-0a4f-4795-8aea-9230e65f712e-kube-api-access-nmcsk\") pod \"sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") " pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:15.179037 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:15.178999    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:15.313734 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:15.313707    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"]
Jul 30 13:54:15.315932 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:54:15.315900    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda992d0b5_0a4f_4795_8aea_9230e65f712e.slice/crio-5d5c40f050a5494a6f0d9f1f0ed1437e59ccc79c5cafc249ed185b1ec314474d WatchSource:0}: Error finding container 5d5c40f050a5494a6f0d9f1f0ed1437e59ccc79c5cafc249ed185b1ec314474d: Status 404 returned error can't find the container with id 5d5c40f050a5494a6f0d9f1f0ed1437e59ccc79c5cafc249ed185b1ec314474d
Jul 30 13:54:16.073180 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:16.073144    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc" event={"ID":"a992d0b5-0a4f-4795-8aea-9230e65f712e","Type":"ContainerStarted","Data":"381cd77de79fb2c8dd13dc6fec5f51f9c4406d34772ad737d3f3b78ccd4947af"}
Jul 30 13:54:16.073575 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:16.073189    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc" event={"ID":"a992d0b5-0a4f-4795-8aea-9230e65f712e","Type":"ContainerStarted","Data":"5d5c40f050a5494a6f0d9f1f0ed1437e59ccc79c5cafc249ed185b1ec314474d"}
Jul 30 13:54:16.091740 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:16.091695    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc" podStartSLOduration=2.091679551 podStartE2EDuration="2.091679551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:54:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:54:16.090888954 +0000 UTC m=+3279.363353350" watchObservedRunningTime="2026-07-30 13:54:16.091679551 +0000 UTC m=+3279.364143948"
Jul 30 13:54:18.740530 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:18.740490    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:54:25.183573 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:25.183539    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:25.184011 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:25.183582    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:25.192717 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:25.192696    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:54:26.136165 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:54:26.136137    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:55:06.358671 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:55:06.358531    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:55:21.939190 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:55:21.939153    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:55:57.832209 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:55:57.832172    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:56:03.418749 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:03.418714    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"]
Jul 30 13:56:04.414046 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:04.414005    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"]
Jul 30 13:56:09.126037 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.126004    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"]
Jul 30 13:56:09.129689 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.129668    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.133046 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.133024    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f4dc0a\"/\"rate-test-model-f4dc0a-kserve-self-signed-certs\""
Jul 30 13:56:09.140193 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.140172    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"]
Jul 30 13:56:09.270217 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.270184    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-tmp-dir\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.270412 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.270232    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrtdx\" (UniqueName: \"kubernetes.io/projected/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-kube-api-access-wrtdx\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.270412 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.270272    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-tls-certs\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.270412 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.270327    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-dshm\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.270412 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.270394    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-home\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.270552 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.270453    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-model-cache\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.371199 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.371162    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-model-cache\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.371375 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.371213    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-tmp-dir\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.371375 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.371330    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrtdx\" (UniqueName: \"kubernetes.io/projected/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-kube-api-access-wrtdx\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.371375 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.371370    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-tls-certs\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.371495 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.371432    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-dshm\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.371495 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.371473    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-home\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.371627 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.371519    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-tmp-dir\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.371693 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.371622    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-model-cache\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.371849 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.371830    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-home\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.373760 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.373735    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-dshm\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.374053 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.374028    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-tls-certs\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.380455 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.380398    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrtdx\" (UniqueName: \"kubernetes.io/projected/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-kube-api-access-wrtdx\") pod \"rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") " pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.440289 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.440262    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:09.576662 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.576634    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"]
Jul 30 13:56:09.579207 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:56:09.579182    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb16e79a0_fd3b_49ab_bf2c_63b8742b40b6.slice/crio-2e21d63488c8d68e15c558a27c3245b011a3453062005e5f0f9d374170a3d74a WatchSource:0}: Error finding container 2e21d63488c8d68e15c558a27c3245b011a3453062005e5f0f9d374170a3d74a: Status 404 returned error can't find the container with id 2e21d63488c8d68e15c558a27c3245b011a3453062005e5f0f9d374170a3d74a
Jul 30 13:56:09.615171 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:09.615148    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs" event={"ID":"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6","Type":"ContainerStarted","Data":"2e21d63488c8d68e15c558a27c3245b011a3453062005e5f0f9d374170a3d74a"}
Jul 30 13:56:10.621144 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:10.621107    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs" event={"ID":"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6","Type":"ContainerStarted","Data":"a65eb2672e976871dad5ada2a9a5003c118f21f8011fd8da2d1db7395cea2bc0"}
Jul 30 13:56:10.640540 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:10.640491    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs" podStartSLOduration=1.640476289 podStartE2EDuration="1.640476289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:56:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:56:10.639270343 +0000 UTC m=+3393.911734740" watchObservedRunningTime="2026-07-30 13:56:10.640476289 +0000 UTC m=+3393.912940686"
Jul 30 13:56:14.627088 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:14.627054    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:56:18.437236 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.437186    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z" podUID="60936de5-432b-4426-9126-ff32e822e3bd" containerName="main" containerID="cri-o://bb94e369a71fd143670ff5927d5e41863b0f262f14946616bbf39a1450c4022b" gracePeriod=45
Jul 30 13:56:18.661743 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.661707    2557 generic.go:356] "Generic (PLEG): container finished" podID="60936de5-432b-4426-9126-ff32e822e3bd" containerID="bb94e369a71fd143670ff5927d5e41863b0f262f14946616bbf39a1450c4022b" exitCode=0
Jul 30 13:56:18.661883 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.661772    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z" event={"ID":"60936de5-432b-4426-9126-ff32e822e3bd","Type":"ContainerDied","Data":"bb94e369a71fd143670ff5927d5e41863b0f262f14946616bbf39a1450c4022b"}
Jul 30 13:56:18.690390 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.690336    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:56:18.867914 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.867880    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-home\") pod \"60936de5-432b-4426-9126-ff32e822e3bd\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") "
Jul 30 13:56:18.867914 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.867919    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-dshm\") pod \"60936de5-432b-4426-9126-ff32e822e3bd\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") "
Jul 30 13:56:18.868129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.867966    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60936de5-432b-4426-9126-ff32e822e3bd-tls-certs\") pod \"60936de5-432b-4426-9126-ff32e822e3bd\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") "
Jul 30 13:56:18.868129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.868005    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z8bq\" (UniqueName: \"kubernetes.io/projected/60936de5-432b-4426-9126-ff32e822e3bd-kube-api-access-7z8bq\") pod \"60936de5-432b-4426-9126-ff32e822e3bd\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") "
Jul 30 13:56:18.868129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.868047    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-model-cache\") pod \"60936de5-432b-4426-9126-ff32e822e3bd\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") "
Jul 30 13:56:18.868129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.868079    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-tmp-dir\") pod \"60936de5-432b-4426-9126-ff32e822e3bd\" (UID: \"60936de5-432b-4426-9126-ff32e822e3bd\") "
Jul 30 13:56:18.868299 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.868164    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-home" (OuterVolumeSpecName: "home") pod "60936de5-432b-4426-9126-ff32e822e3bd" (UID: "60936de5-432b-4426-9126-ff32e822e3bd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:56:18.868370 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.868349    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-model-cache" (OuterVolumeSpecName: "model-cache") pod "60936de5-432b-4426-9126-ff32e822e3bd" (UID: "60936de5-432b-4426-9126-ff32e822e3bd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:56:18.868438 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.868374    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "60936de5-432b-4426-9126-ff32e822e3bd" (UID: "60936de5-432b-4426-9126-ff32e822e3bd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:56:18.868493 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.868442    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-home\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:56:18.868493 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.868453    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-model-cache\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:56:18.868493 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.868463    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-tmp-dir\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:56:18.870197 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.870173    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-dshm" (OuterVolumeSpecName: "dshm") pod "60936de5-432b-4426-9126-ff32e822e3bd" (UID: "60936de5-432b-4426-9126-ff32e822e3bd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:56:18.870197 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.870191    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60936de5-432b-4426-9126-ff32e822e3bd-kube-api-access-7z8bq" (OuterVolumeSpecName: "kube-api-access-7z8bq") pod "60936de5-432b-4426-9126-ff32e822e3bd" (UID: "60936de5-432b-4426-9126-ff32e822e3bd"). InnerVolumeSpecName "kube-api-access-7z8bq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:56:18.870360 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.870212    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60936de5-432b-4426-9126-ff32e822e3bd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "60936de5-432b-4426-9126-ff32e822e3bd" (UID: "60936de5-432b-4426-9126-ff32e822e3bd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 13:56:18.969042 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.968960    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60936de5-432b-4426-9126-ff32e822e3bd-dshm\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:56:18.969042 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.968988    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60936de5-432b-4426-9126-ff32e822e3bd-tls-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:56:18.969042 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:18.969000    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z8bq\" (UniqueName: \"kubernetes.io/projected/60936de5-432b-4426-9126-ff32e822e3bd-kube-api-access-7z8bq\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:56:19.433505 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.433438    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc" podUID="a992d0b5-0a4f-4795-8aea-9230e65f712e" containerName="main" containerID="cri-o://381cd77de79fb2c8dd13dc6fec5f51f9c4406d34772ad737d3f3b78ccd4947af" gracePeriod=45
Jul 30 13:56:19.440959 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.440927    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:19.441291 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.440980    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:19.453808 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.453787    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:19.668527 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.668502    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"
Jul 30 13:56:19.668703 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.668530    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z" event={"ID":"60936de5-432b-4426-9126-ff32e822e3bd","Type":"ContainerDied","Data":"a18fe0a15f433cae9518fb55f4695dab361a45b26a52f9c8b16b63614694b2f6"}
Jul 30 13:56:19.668703 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.668583    2557 scope.go:117] "RemoveContainer" containerID="bb94e369a71fd143670ff5927d5e41863b0f262f14946616bbf39a1450c4022b"
Jul 30 13:56:19.669998 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.669976    2557 generic.go:356] "Generic (PLEG): container finished" podID="a992d0b5-0a4f-4795-8aea-9230e65f712e" containerID="381cd77de79fb2c8dd13dc6fec5f51f9c4406d34772ad737d3f3b78ccd4947af" exitCode=0
Jul 30 13:56:19.670087 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.670046    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc" event={"ID":"a992d0b5-0a4f-4795-8aea-9230e65f712e","Type":"ContainerDied","Data":"381cd77de79fb2c8dd13dc6fec5f51f9c4406d34772ad737d3f3b78ccd4947af"}
Jul 30 13:56:19.682636 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.682614    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:56:19.689640 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.689622    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:56:19.732454 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.732425    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"]
Jul 30 13:56:19.738343 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.738316    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/sub-test-model-f4dc0a-kserve-6d77c58c99-bcr6z"]
Jul 30 13:56:19.877233 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.877198    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a992d0b5-0a4f-4795-8aea-9230e65f712e-tls-certs\") pod \"a992d0b5-0a4f-4795-8aea-9230e65f712e\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") "
Jul 30 13:56:19.877404 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.877257    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmcsk\" (UniqueName: \"kubernetes.io/projected/a992d0b5-0a4f-4795-8aea-9230e65f712e-kube-api-access-nmcsk\") pod \"a992d0b5-0a4f-4795-8aea-9230e65f712e\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") "
Jul 30 13:56:19.877404 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.877313    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-home\") pod \"a992d0b5-0a4f-4795-8aea-9230e65f712e\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") "
Jul 30 13:56:19.877404 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.877341    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-tmp-dir\") pod \"a992d0b5-0a4f-4795-8aea-9230e65f712e\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") "
Jul 30 13:56:19.877404 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.877366    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-dshm\") pod \"a992d0b5-0a4f-4795-8aea-9230e65f712e\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") "
Jul 30 13:56:19.877669 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.877412    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-model-cache\") pod \"a992d0b5-0a4f-4795-8aea-9230e65f712e\" (UID: \"a992d0b5-0a4f-4795-8aea-9230e65f712e\") "
Jul 30 13:56:19.877669 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.877523    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-home" (OuterVolumeSpecName: "home") pod "a992d0b5-0a4f-4795-8aea-9230e65f712e" (UID: "a992d0b5-0a4f-4795-8aea-9230e65f712e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:56:19.877669 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.877624    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a992d0b5-0a4f-4795-8aea-9230e65f712e" (UID: "a992d0b5-0a4f-4795-8aea-9230e65f712e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:56:19.877827 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.877701    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-model-cache" (OuterVolumeSpecName: "model-cache") pod "a992d0b5-0a4f-4795-8aea-9230e65f712e" (UID: "a992d0b5-0a4f-4795-8aea-9230e65f712e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:56:19.877827 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.877753    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-home\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:56:19.877827 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.877767    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-tmp-dir\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:56:19.879451 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.879426    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a992d0b5-0a4f-4795-8aea-9230e65f712e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a992d0b5-0a4f-4795-8aea-9230e65f712e" (UID: "a992d0b5-0a4f-4795-8aea-9230e65f712e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 13:56:19.879561 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.879483    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a992d0b5-0a4f-4795-8aea-9230e65f712e-kube-api-access-nmcsk" (OuterVolumeSpecName: "kube-api-access-nmcsk") pod "a992d0b5-0a4f-4795-8aea-9230e65f712e" (UID: "a992d0b5-0a4f-4795-8aea-9230e65f712e"). InnerVolumeSpecName "kube-api-access-nmcsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:56:19.879561 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.879507    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-dshm" (OuterVolumeSpecName: "dshm") pod "a992d0b5-0a4f-4795-8aea-9230e65f712e" (UID: "a992d0b5-0a4f-4795-8aea-9230e65f712e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:56:19.978879 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.978805    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a992d0b5-0a4f-4795-8aea-9230e65f712e-tls-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:56:19.978879 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.978830    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmcsk\" (UniqueName: \"kubernetes.io/projected/a992d0b5-0a4f-4795-8aea-9230e65f712e-kube-api-access-nmcsk\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:56:19.978879 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.978840    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-dshm\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:56:19.978879 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:19.978850    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a992d0b5-0a4f-4795-8aea-9230e65f712e-model-cache\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:56:20.675771 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:20.675737    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"
Jul 30 13:56:20.676266 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:20.675778    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc" event={"ID":"a992d0b5-0a4f-4795-8aea-9230e65f712e","Type":"ContainerDied","Data":"5d5c40f050a5494a6f0d9f1f0ed1437e59ccc79c5cafc249ed185b1ec314474d"}
Jul 30 13:56:20.676266 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:20.675838    2557 scope.go:117] "RemoveContainer" containerID="381cd77de79fb2c8dd13dc6fec5f51f9c4406d34772ad737d3f3b78ccd4947af"
Jul 30 13:56:20.697715 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:20.697689    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"]
Jul 30 13:56:20.701733 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:20.701702    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/sub-test-model-e945a1-kserve-7f565f5dc7-6n9kc"]
Jul 30 13:56:21.183268 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:21.183237    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60936de5-432b-4426-9126-ff32e822e3bd" path="/var/lib/kubelet/pods/60936de5-432b-4426-9126-ff32e822e3bd/volumes"
Jul 30 13:56:21.183641 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:21.183625    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a992d0b5-0a4f-4795-8aea-9230e65f712e" path="/var/lib/kubelet/pods/a992d0b5-0a4f-4795-8aea-9230e65f712e/volumes"
Jul 30 13:56:36.740333 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.740248    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"]
Jul 30 13:56:36.740932 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.740796    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60936de5-432b-4426-9126-ff32e822e3bd" containerName="main"
Jul 30 13:56:36.740932 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.740816    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="60936de5-432b-4426-9126-ff32e822e3bd" containerName="main"
Jul 30 13:56:36.740932 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.740863    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a992d0b5-0a4f-4795-8aea-9230e65f712e" containerName="main"
Jul 30 13:56:36.740932 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.740872    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a992d0b5-0a4f-4795-8aea-9230e65f712e" containerName="main"
Jul 30 13:56:36.741158 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.740963    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a992d0b5-0a4f-4795-8aea-9230e65f712e" containerName="main"
Jul 30 13:56:36.741158 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.740976    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="60936de5-432b-4426-9126-ff32e822e3bd" containerName="main"
Jul 30 13:56:36.745373 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.745354    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.750078 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.750055    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e945a1\"/\"kube-root-ca.crt\""
Jul 30 13:56:36.750451 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.750291    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e945a1\"/\"default-dockercfg-jbtzb\""
Jul 30 13:56:36.750451 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.750292    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e945a1\"/\"rate-test-model-e945a1-kserve-self-signed-certs\""
Jul 30 13:56:36.750451 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.750334    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e945a1\"/\"openshift-service-ca.crt\""
Jul 30 13:56:36.753320 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.753294    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"]
Jul 30 13:56:36.815432 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.815397    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-dshm\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.815617 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.815482    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-model-cache\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.815617 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.815518    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc2w5\" (UniqueName: \"kubernetes.io/projected/b969d9a4-8d43-4257-baee-a1259728a79e-kube-api-access-nc2w5\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.815617 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.815548    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-tmp-dir\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.815617 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.815573    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-home\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.815617 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.815591    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b969d9a4-8d43-4257-baee-a1259728a79e-tls-certs\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.916448 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.916410    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-model-cache\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.916679 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.916466    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc2w5\" (UniqueName: \"kubernetes.io/projected/b969d9a4-8d43-4257-baee-a1259728a79e-kube-api-access-nc2w5\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.916679 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.916500    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-tmp-dir\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.916679 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.916533    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-home\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.916679 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.916660    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b969d9a4-8d43-4257-baee-a1259728a79e-tls-certs\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.916933 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.916714    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-dshm\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.916933 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.916872    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-model-cache\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.917203 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.917175    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-home\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.917203 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.917195    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-tmp-dir\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.918936 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.918916    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-dshm\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.919310 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.919288    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b969d9a4-8d43-4257-baee-a1259728a79e-tls-certs\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:36.925762 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:36.925739    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc2w5\" (UniqueName: \"kubernetes.io/projected/b969d9a4-8d43-4257-baee-a1259728a79e-kube-api-access-nc2w5\") pod \"rate-test-model-e945a1-kserve-86ddf757f-89584\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") " pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:37.056973 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:37.056893    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:37.191710 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:37.191680    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"]
Jul 30 13:56:37.192292 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:56:37.192263    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb969d9a4_8d43_4257_baee_a1259728a79e.slice/crio-5c0b0f4931576ee424a2084bdadf9a87e92d6c8c82e300e37088a26da5423cda WatchSource:0}: Error finding container 5c0b0f4931576ee424a2084bdadf9a87e92d6c8c82e300e37088a26da5423cda: Status 404 returned error can't find the container with id 5c0b0f4931576ee424a2084bdadf9a87e92d6c8c82e300e37088a26da5423cda
Jul 30 13:56:37.763171 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:37.763130    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584" event={"ID":"b969d9a4-8d43-4257-baee-a1259728a79e","Type":"ContainerStarted","Data":"445e0fc1596dc9f251f945e77ff312e5bc330dcb6367d8037670491da8164c33"}
Jul 30 13:56:37.763552 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:37.763182    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584" event={"ID":"b969d9a4-8d43-4257-baee-a1259728a79e","Type":"ContainerStarted","Data":"5c0b0f4931576ee424a2084bdadf9a87e92d6c8c82e300e37088a26da5423cda"}
Jul 30 13:56:37.783994 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:37.783943    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584" podStartSLOduration=1.78392887 podStartE2EDuration="1.78392887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:56:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:56:37.780856741 +0000 UTC m=+3421.053321137" watchObservedRunningTime="2026-07-30 13:56:37.78392887 +0000 UTC m=+3421.056393266"
Jul 30 13:56:41.134413 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:41.134380    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:56:47.057989 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:47.057960    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:47.058384 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:47.058002    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:47.070307 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:47.070285    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:56:47.819803 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:56:47.819775    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:57:33.246090 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:57:33.246061    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:57:47.835242 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:57:47.835210    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:58:20.335719 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:20.335683    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:58:26.596238 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:26.596206    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"]
Jul 30 13:58:27.530526 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:27.530497    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"]
Jul 30 13:58:28.508024 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:28.507990    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7483-140093674277040"]
Jul 30 13:58:28.511310 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:28.511293    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674277040"
Jul 30 13:58:28.514815 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:28.514790    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8njnd\""
Jul 30 13:58:28.516533 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:28.516511    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140093674277040"]
Jul 30 13:58:28.652636 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:28.652586    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z44x\" (UniqueName: \"kubernetes.io/projected/d42c162e-9ef5-42c6-9b01-d3f600141787-kube-api-access-8z44x\") pod \"test-curl-7483-140093674277040\" (UID: \"d42c162e-9ef5-42c6-9b01-d3f600141787\") " pod="opendatahub/test-curl-7483-140093674277040"
Jul 30 13:58:28.753437 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:28.753400    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z44x\" (UniqueName: \"kubernetes.io/projected/d42c162e-9ef5-42c6-9b01-d3f600141787-kube-api-access-8z44x\") pod \"test-curl-7483-140093674277040\" (UID: \"d42c162e-9ef5-42c6-9b01-d3f600141787\") " pod="opendatahub/test-curl-7483-140093674277040"
Jul 30 13:58:28.762712 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:28.762643    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z44x\" (UniqueName: \"kubernetes.io/projected/d42c162e-9ef5-42c6-9b01-d3f600141787-kube-api-access-8z44x\") pod \"test-curl-7483-140093674277040\" (UID: \"d42c162e-9ef5-42c6-9b01-d3f600141787\") " pod="opendatahub/test-curl-7483-140093674277040"
Jul 30 13:58:28.823024 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:28.822990    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674277040"
Jul 30 13:58:28.952075 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:28.952050    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140093674277040"]
Jul 30 13:58:28.954033 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:58:28.954005    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd42c162e_9ef5_42c6_9b01_d3f600141787.slice/crio-439ca53d1bc04a505e0a4941b3f67bf310c72df08586237b669c2547f4f51551 WatchSource:0}: Error finding container 439ca53d1bc04a505e0a4941b3f67bf310c72df08586237b669c2547f4f51551: Status 404 returned error can't find the container with id 439ca53d1bc04a505e0a4941b3f67bf310c72df08586237b669c2547f4f51551
Jul 30 13:58:28.955301 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:28.955281    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 13:58:29.292777 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:29.292727    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140093674277040" event={"ID":"d42c162e-9ef5-42c6-9b01-d3f600141787","Type":"ContainerStarted","Data":"439ca53d1bc04a505e0a4941b3f67bf310c72df08586237b669c2547f4f51551"}
Jul 30 13:58:30.299122 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:30.299087    2557 generic.go:356] "Generic (PLEG): container finished" podID="d42c162e-9ef5-42c6-9b01-d3f600141787" containerID="6a1a5a79483229e9ae2453d72ac697b5cb9e616a136f46a385baed3e69940b03" exitCode=0
Jul 30 13:58:30.299512 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:30.299139    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140093674277040" event={"ID":"d42c162e-9ef5-42c6-9b01-d3f600141787","Type":"ContainerDied","Data":"6a1a5a79483229e9ae2453d72ac697b5cb9e616a136f46a385baed3e69940b03"}
Jul 30 13:58:31.435965 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.435941    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674277040"
Jul 30 13:58:31.471415 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.471389    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7483-140093674277040_d42c162e-9ef5-42c6-9b01-d3f600141787/test-curl-7483-140093674277040/0.log"
Jul 30 13:58:31.498099 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.498069    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7483-140093674277040"]
Jul 30 13:58:31.503354 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.503334    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7483-140093674277040"]
Jul 30 13:58:31.579140 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.579061    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z44x\" (UniqueName: \"kubernetes.io/projected/d42c162e-9ef5-42c6-9b01-d3f600141787-kube-api-access-8z44x\") pod \"d42c162e-9ef5-42c6-9b01-d3f600141787\" (UID: \"d42c162e-9ef5-42c6-9b01-d3f600141787\") "
Jul 30 13:58:31.581294 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.581266    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d42c162e-9ef5-42c6-9b01-d3f600141787-kube-api-access-8z44x" (OuterVolumeSpecName: "kube-api-access-8z44x") pod "d42c162e-9ef5-42c6-9b01-d3f600141787" (UID: "d42c162e-9ef5-42c6-9b01-d3f600141787"). InnerVolumeSpecName "kube-api-access-8z44x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:58:31.625287 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.625261    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7483-140093674276528"]
Jul 30 13:58:31.625629 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.625617    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d42c162e-9ef5-42c6-9b01-d3f600141787" containerName="test-curl-7483-140093674277040"
Jul 30 13:58:31.625678 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.625631    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d42c162e-9ef5-42c6-9b01-d3f600141787" containerName="test-curl-7483-140093674277040"
Jul 30 13:58:31.625714 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.625703    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d42c162e-9ef5-42c6-9b01-d3f600141787" containerName="test-curl-7483-140093674277040"
Jul 30 13:58:31.628715 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.628701    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674276528"
Jul 30 13:58:31.633083 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.633061    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140093674276528"]
Jul 30 13:58:31.680520 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.680482    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z44x\" (UniqueName: \"kubernetes.io/projected/d42c162e-9ef5-42c6-9b01-d3f600141787-kube-api-access-8z44x\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:31.781376 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.781343    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjncc\" (UniqueName: \"kubernetes.io/projected/243e1c5f-cfc4-417b-b152-dbef7cb4813c-kube-api-access-cjncc\") pod \"test-curl-7483-140093674276528\" (UID: \"243e1c5f-cfc4-417b-b152-dbef7cb4813c\") " pod="opendatahub/test-curl-7483-140093674276528"
Jul 30 13:58:31.882280 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.882169    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjncc\" (UniqueName: \"kubernetes.io/projected/243e1c5f-cfc4-417b-b152-dbef7cb4813c-kube-api-access-cjncc\") pod \"test-curl-7483-140093674276528\" (UID: \"243e1c5f-cfc4-417b-b152-dbef7cb4813c\") " pod="opendatahub/test-curl-7483-140093674276528"
Jul 30 13:58:31.893435 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.893410    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjncc\" (UniqueName: \"kubernetes.io/projected/243e1c5f-cfc4-417b-b152-dbef7cb4813c-kube-api-access-cjncc\") pod \"test-curl-7483-140093674276528\" (UID: \"243e1c5f-cfc4-417b-b152-dbef7cb4813c\") " pod="opendatahub/test-curl-7483-140093674276528"
Jul 30 13:58:31.939075 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:31.939052    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674276528"
Jul 30 13:58:32.071764 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:32.071738    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140093674276528"]
Jul 30 13:58:32.073942 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:58:32.073916    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod243e1c5f_cfc4_417b_b152_dbef7cb4813c.slice/crio-30f17c90dfec3af4e46e7d586f2dbd850582cc73795c5cc3da42b929263e88ad WatchSource:0}: Error finding container 30f17c90dfec3af4e46e7d586f2dbd850582cc73795c5cc3da42b929263e88ad: Status 404 returned error can't find the container with id 30f17c90dfec3af4e46e7d586f2dbd850582cc73795c5cc3da42b929263e88ad
Jul 30 13:58:32.310310 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:32.310281    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674277040"
Jul 30 13:58:32.310496 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:32.310333    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="439ca53d1bc04a505e0a4941b3f67bf310c72df08586237b669c2547f4f51551"
Jul 30 13:58:32.311515 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:32.311488    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140093674276528" event={"ID":"243e1c5f-cfc4-417b-b152-dbef7cb4813c","Type":"ContainerStarted","Data":"30f17c90dfec3af4e46e7d586f2dbd850582cc73795c5cc3da42b929263e88ad"}
Jul 30 13:58:32.321457 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:58:32.321428    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7483-140093674277040\" is forbidden: User \"system:node:ip-10-0-143-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-33.ec2.internal' and this object" podUID="d42c162e-9ef5-42c6-9b01-d3f600141787" pod="opendatahub/test-curl-7483-140093674277040"
Jul 30 13:58:33.183935 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:33.183902    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d42c162e-9ef5-42c6-9b01-d3f600141787" path="/var/lib/kubelet/pods/d42c162e-9ef5-42c6-9b01-d3f600141787/volumes"
Jul 30 13:58:33.318013 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:33.317973    2557 generic.go:356] "Generic (PLEG): container finished" podID="243e1c5f-cfc4-417b-b152-dbef7cb4813c" containerID="0bc3570e21025ed9430f0b2bee471b9442a28ab6d5e7e2fd89d02101cb866eb6" exitCode=0
Jul 30 13:58:33.318191 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:33.318060    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140093674276528" event={"ID":"243e1c5f-cfc4-417b-b152-dbef7cb4813c","Type":"ContainerDied","Data":"0bc3570e21025ed9430f0b2bee471b9442a28ab6d5e7e2fd89d02101cb866eb6"}
Jul 30 13:58:34.454970 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.454949    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674276528"
Jul 30 13:58:34.476586 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.476561    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7483-140093674276528_243e1c5f-cfc4-417b-b152-dbef7cb4813c/test-curl-7483-140093674276528/0.log"
Jul 30 13:58:34.499612 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.499577    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7483-140093674276528"]
Jul 30 13:58:34.503271 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.503248    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7483-140093674276528"]
Jul 30 13:58:34.606566 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.606480    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjncc\" (UniqueName: \"kubernetes.io/projected/243e1c5f-cfc4-417b-b152-dbef7cb4813c-kube-api-access-cjncc\") pod \"243e1c5f-cfc4-417b-b152-dbef7cb4813c\" (UID: \"243e1c5f-cfc4-417b-b152-dbef7cb4813c\") "
Jul 30 13:58:34.608539 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.608513    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/243e1c5f-cfc4-417b-b152-dbef7cb4813c-kube-api-access-cjncc" (OuterVolumeSpecName: "kube-api-access-cjncc") pod "243e1c5f-cfc4-417b-b152-dbef7cb4813c" (UID: "243e1c5f-cfc4-417b-b152-dbef7cb4813c"). InnerVolumeSpecName "kube-api-access-cjncc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:58:34.665166 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.665133    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7483-140093674462352"]
Jul 30 13:58:34.665521 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.665507    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="243e1c5f-cfc4-417b-b152-dbef7cb4813c" containerName="test-curl-7483-140093674276528"
Jul 30 13:58:34.665568 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.665522    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="243e1c5f-cfc4-417b-b152-dbef7cb4813c" containerName="test-curl-7483-140093674276528"
Jul 30 13:58:34.665639 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.665587    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="243e1c5f-cfc4-417b-b152-dbef7cb4813c" containerName="test-curl-7483-140093674276528"
Jul 30 13:58:34.668413 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.668398    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674462352"
Jul 30 13:58:34.675747 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.675718    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140093674462352"]
Jul 30 13:58:34.707832 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.707801    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjncc\" (UniqueName: \"kubernetes.io/projected/243e1c5f-cfc4-417b-b152-dbef7cb4813c-kube-api-access-cjncc\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:34.809029 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.808996    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nxc2\" (UniqueName: \"kubernetes.io/projected/8dfc3dd4-95f2-4688-8a58-37fc721ba9e1-kube-api-access-5nxc2\") pod \"test-curl-7483-140093674462352\" (UID: \"8dfc3dd4-95f2-4688-8a58-37fc721ba9e1\") " pod="opendatahub/test-curl-7483-140093674462352"
Jul 30 13:58:34.910436 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.910403    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nxc2\" (UniqueName: \"kubernetes.io/projected/8dfc3dd4-95f2-4688-8a58-37fc721ba9e1-kube-api-access-5nxc2\") pod \"test-curl-7483-140093674462352\" (UID: \"8dfc3dd4-95f2-4688-8a58-37fc721ba9e1\") " pod="opendatahub/test-curl-7483-140093674462352"
Jul 30 13:58:34.918536 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.918506    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nxc2\" (UniqueName: \"kubernetes.io/projected/8dfc3dd4-95f2-4688-8a58-37fc721ba9e1-kube-api-access-5nxc2\") pod \"test-curl-7483-140093674462352\" (UID: \"8dfc3dd4-95f2-4688-8a58-37fc721ba9e1\") " pod="opendatahub/test-curl-7483-140093674462352"
Jul 30 13:58:34.978741 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:34.978705    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674462352"
Jul 30 13:58:35.110018 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:35.109969    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140093674462352"]
Jul 30 13:58:35.112272 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:58:35.112242    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8dfc3dd4_95f2_4688_8a58_37fc721ba9e1.slice/crio-c204a4f50556070ebe10d0fcfab9e85bfba1af94ca5f05c5cde3b77aeafbcf2d WatchSource:0}: Error finding container c204a4f50556070ebe10d0fcfab9e85bfba1af94ca5f05c5cde3b77aeafbcf2d: Status 404 returned error can't find the container with id c204a4f50556070ebe10d0fcfab9e85bfba1af94ca5f05c5cde3b77aeafbcf2d
Jul 30 13:58:35.185236 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:35.185157    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="243e1c5f-cfc4-417b-b152-dbef7cb4813c" path="/var/lib/kubelet/pods/243e1c5f-cfc4-417b-b152-dbef7cb4813c/volumes"
Jul 30 13:58:35.331099 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:35.331068    2557 scope.go:117] "RemoveContainer" containerID="0bc3570e21025ed9430f0b2bee471b9442a28ab6d5e7e2fd89d02101cb866eb6"
Jul 30 13:58:35.331292 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:35.331070    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674276528"
Jul 30 13:58:35.332376 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:35.332288    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140093674462352" event={"ID":"8dfc3dd4-95f2-4688-8a58-37fc721ba9e1","Type":"ContainerStarted","Data":"c204a4f50556070ebe10d0fcfab9e85bfba1af94ca5f05c5cde3b77aeafbcf2d"}
Jul 30 13:58:36.340527 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:36.340496    2557 generic.go:356] "Generic (PLEG): container finished" podID="8dfc3dd4-95f2-4688-8a58-37fc721ba9e1" containerID="7f204b372b2ba3901719a5044664c18ddbdead3867d29284588dd5fc2f17f525" exitCode=0
Jul 30 13:58:36.340929 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:36.340577    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140093674462352" event={"ID":"8dfc3dd4-95f2-4688-8a58-37fc721ba9e1","Type":"ContainerDied","Data":"7f204b372b2ba3901719a5044664c18ddbdead3867d29284588dd5fc2f17f525"}
Jul 30 13:58:37.479209 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.479184    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674462352"
Jul 30 13:58:37.499675 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.499639    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7483-140093674462352_8dfc3dd4-95f2-4688-8a58-37fc721ba9e1/test-curl-7483-140093674462352/0.log"
Jul 30 13:58:37.524196 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.524167    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7483-140093674462352"]
Jul 30 13:58:37.527621 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.527579    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7483-140093674462352"]
Jul 30 13:58:37.639128 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.639104    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nxc2\" (UniqueName: \"kubernetes.io/projected/8dfc3dd4-95f2-4688-8a58-37fc721ba9e1-kube-api-access-5nxc2\") pod \"8dfc3dd4-95f2-4688-8a58-37fc721ba9e1\" (UID: \"8dfc3dd4-95f2-4688-8a58-37fc721ba9e1\") "
Jul 30 13:58:37.641070 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.641048    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dfc3dd4-95f2-4688-8a58-37fc721ba9e1-kube-api-access-5nxc2" (OuterVolumeSpecName: "kube-api-access-5nxc2") pod "8dfc3dd4-95f2-4688-8a58-37fc721ba9e1" (UID: "8dfc3dd4-95f2-4688-8a58-37fc721ba9e1"). InnerVolumeSpecName "kube-api-access-5nxc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:58:37.650635 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.650614    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7483-140093674462352"]
Jul 30 13:58:37.650998 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.650987    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dfc3dd4-95f2-4688-8a58-37fc721ba9e1" containerName="test-curl-7483-140093674462352"
Jul 30 13:58:37.651042 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.651001    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dfc3dd4-95f2-4688-8a58-37fc721ba9e1" containerName="test-curl-7483-140093674462352"
Jul 30 13:58:37.651078 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.651054    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dfc3dd4-95f2-4688-8a58-37fc721ba9e1" containerName="test-curl-7483-140093674462352"
Jul 30 13:58:37.655075 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.655061    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674462352"
Jul 30 13:58:37.659990 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.659967    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140093674462352"]
Jul 30 13:58:37.734367 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.734333    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:58:37.740109 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.740086    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nxc2\" (UniqueName: \"kubernetes.io/projected/8dfc3dd4-95f2-4688-8a58-37fc721ba9e1-kube-api-access-5nxc2\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:37.841442 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.841409    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6x65\" (UniqueName: \"kubernetes.io/projected/a44599a7-c3fa-40c4-a05a-674c15cdfc31-kube-api-access-f6x65\") pod \"test-curl-7483-140093674462352\" (UID: \"a44599a7-c3fa-40c4-a05a-674c15cdfc31\") " pod="opendatahub/test-curl-7483-140093674462352"
Jul 30 13:58:37.942621 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.942568    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6x65\" (UniqueName: \"kubernetes.io/projected/a44599a7-c3fa-40c4-a05a-674c15cdfc31-kube-api-access-f6x65\") pod \"test-curl-7483-140093674462352\" (UID: \"a44599a7-c3fa-40c4-a05a-674c15cdfc31\") " pod="opendatahub/test-curl-7483-140093674462352"
Jul 30 13:58:37.951438 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.951416    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6x65\" (UniqueName: \"kubernetes.io/projected/a44599a7-c3fa-40c4-a05a-674c15cdfc31-kube-api-access-f6x65\") pod \"test-curl-7483-140093674462352\" (UID: \"a44599a7-c3fa-40c4-a05a-674c15cdfc31\") " pod="opendatahub/test-curl-7483-140093674462352"
Jul 30 13:58:37.965330 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:37.965305    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674462352"
Jul 30 13:58:38.100751 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:38.100685    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140093674462352"]
Jul 30 13:58:38.102832 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:58:38.102805    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda44599a7_c3fa_40c4_a05a_674c15cdfc31.slice/crio-cf1a0180e897993566da18c5215e2e9c3162945a592c375bebc2de89a5487803 WatchSource:0}: Error finding container cf1a0180e897993566da18c5215e2e9c3162945a592c375bebc2de89a5487803: Status 404 returned error can't find the container with id cf1a0180e897993566da18c5215e2e9c3162945a592c375bebc2de89a5487803
Jul 30 13:58:38.350569 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:38.350469    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140093674462352" event={"ID":"a44599a7-c3fa-40c4-a05a-674c15cdfc31","Type":"ContainerStarted","Data":"cf1a0180e897993566da18c5215e2e9c3162945a592c375bebc2de89a5487803"}
Jul 30 13:58:38.351621 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:38.351583    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c204a4f50556070ebe10d0fcfab9e85bfba1af94ca5f05c5cde3b77aeafbcf2d"
Jul 30 13:58:38.351621 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:38.351617    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674462352"
Jul 30 13:58:38.364236 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:38.364206    2557 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7483-140093674462352" oldPodUID="8dfc3dd4-95f2-4688-8a58-37fc721ba9e1" podUID="a44599a7-c3fa-40c4-a05a-674c15cdfc31"
Jul 30 13:58:39.183370 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:39.183335    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dfc3dd4-95f2-4688-8a58-37fc721ba9e1" path="/var/lib/kubelet/pods/8dfc3dd4-95f2-4688-8a58-37fc721ba9e1/volumes"
Jul 30 13:58:39.359319 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:39.359291    2557 generic.go:356] "Generic (PLEG): container finished" podID="a44599a7-c3fa-40c4-a05a-674c15cdfc31" containerID="4c7310195cf08c4b8e353dbb1b3375b5d67a8812a2488de4ef2b495866de34ac" exitCode=0
Jul 30 13:58:39.359463 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:39.359377    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140093674462352" event={"ID":"a44599a7-c3fa-40c4-a05a-674c15cdfc31","Type":"ContainerDied","Data":"4c7310195cf08c4b8e353dbb1b3375b5d67a8812a2488de4ef2b495866de34ac"}
Jul 30 13:58:40.495854 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.495823    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674462352"
Jul 30 13:58:40.517898 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.517867    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7483-140093674462352_a44599a7-c3fa-40c4-a05a-674c15cdfc31/test-curl-7483-140093674462352/0.log"
Jul 30 13:58:40.543194 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.543161    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7483-140093674462352"]
Jul 30 13:58:40.544898 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.544879    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7483-140093674462352"]
Jul 30 13:58:40.668577 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.668543    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6x65\" (UniqueName: \"kubernetes.io/projected/a44599a7-c3fa-40c4-a05a-674c15cdfc31-kube-api-access-f6x65\") pod \"a44599a7-c3fa-40c4-a05a-674c15cdfc31\" (UID: \"a44599a7-c3fa-40c4-a05a-674c15cdfc31\") "
Jul 30 13:58:40.670560 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.670537    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a44599a7-c3fa-40c4-a05a-674c15cdfc31-kube-api-access-f6x65" (OuterVolumeSpecName: "kube-api-access-f6x65") pod "a44599a7-c3fa-40c4-a05a-674c15cdfc31" (UID: "a44599a7-c3fa-40c4-a05a-674c15cdfc31"). InnerVolumeSpecName "kube-api-access-f6x65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:58:40.675223 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.675198    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7483-140093674254384"]
Jul 30 13:58:40.675562 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.675548    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a44599a7-c3fa-40c4-a05a-674c15cdfc31" containerName="test-curl-7483-140093674462352"
Jul 30 13:58:40.675562 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.675562    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a44599a7-c3fa-40c4-a05a-674c15cdfc31" containerName="test-curl-7483-140093674462352"
Jul 30 13:58:40.675698 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.675645    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a44599a7-c3fa-40c4-a05a-674c15cdfc31" containerName="test-curl-7483-140093674462352"
Jul 30 13:58:40.678512 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.678493    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674254384"
Jul 30 13:58:40.684910 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.684858    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140093674254384"]
Jul 30 13:58:40.770282 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.770249    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6x65\" (UniqueName: \"kubernetes.io/projected/a44599a7-c3fa-40c4-a05a-674c15cdfc31-kube-api-access-f6x65\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:40.871031 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.870998    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc657\" (UniqueName: \"kubernetes.io/projected/ddd6202d-1741-4084-afa9-046fb6c96d72-kube-api-access-vc657\") pod \"test-curl-7483-140093674254384\" (UID: \"ddd6202d-1741-4084-afa9-046fb6c96d72\") " pod="opendatahub/test-curl-7483-140093674254384"
Jul 30 13:58:40.972696 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.972585    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc657\" (UniqueName: \"kubernetes.io/projected/ddd6202d-1741-4084-afa9-046fb6c96d72-kube-api-access-vc657\") pod \"test-curl-7483-140093674254384\" (UID: \"ddd6202d-1741-4084-afa9-046fb6c96d72\") " pod="opendatahub/test-curl-7483-140093674254384"
Jul 30 13:58:40.981692 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.981658    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc657\" (UniqueName: \"kubernetes.io/projected/ddd6202d-1741-4084-afa9-046fb6c96d72-kube-api-access-vc657\") pod \"test-curl-7483-140093674254384\" (UID: \"ddd6202d-1741-4084-afa9-046fb6c96d72\") " pod="opendatahub/test-curl-7483-140093674254384"
Jul 30 13:58:40.987896 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:40.987876    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674254384"
Jul 30 13:58:41.122194 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.122171    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140093674254384"]
Jul 30 13:58:41.123944 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:58:41.123917    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddd6202d_1741_4084_afa9_046fb6c96d72.slice/crio-60063fce7f192d37e32e085cbe5f16561fec850654eb1605c37ef3527682635b WatchSource:0}: Error finding container 60063fce7f192d37e32e085cbe5f16561fec850654eb1605c37ef3527682635b: Status 404 returned error can't find the container with id 60063fce7f192d37e32e085cbe5f16561fec850654eb1605c37ef3527682635b
Jul 30 13:58:41.183099 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.183071    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a44599a7-c3fa-40c4-a05a-674c15cdfc31" path="/var/lib/kubelet/pods/a44599a7-c3fa-40c4-a05a-674c15cdfc31/volumes"
Jul 30 13:58:41.371166 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.371077    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140093674254384" event={"ID":"ddd6202d-1741-4084-afa9-046fb6c96d72","Type":"ContainerStarted","Data":"60063fce7f192d37e32e085cbe5f16561fec850654eb1605c37ef3527682635b"}
Jul 30 13:58:41.372492 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.372474    2557 scope.go:117] "RemoveContainer" containerID="4c7310195cf08c4b8e353dbb1b3375b5d67a8812a2488de4ef2b495866de34ac"
Jul 30 13:58:41.372492 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.372480    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674462352"
Jul 30 13:58:41.614501 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.614460    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs" podUID="b16e79a0-fd3b-49ab-bf2c-63b8742b40b6" containerName="main" containerID="cri-o://a65eb2672e976871dad5ada2a9a5003c118f21f8011fd8da2d1db7395cea2bc0" gracePeriod=45
Jul 30 13:58:41.858620 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.858582    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:58:41.889496 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.889420    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrtdx\" (UniqueName: \"kubernetes.io/projected/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-kube-api-access-wrtdx\") pod \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") "
Jul 30 13:58:41.889496 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.889467    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-home\") pod \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") "
Jul 30 13:58:41.889727 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.889569    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-dshm\") pod \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") "
Jul 30 13:58:41.889727 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.889625    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-tmp-dir\") pod \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") "
Jul 30 13:58:41.889727 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.889665    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-model-cache\") pod \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") "
Jul 30 13:58:41.889727 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.889702    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-tls-certs\") pod \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\" (UID: \"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6\") "
Jul 30 13:58:41.889935 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.889789    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-home" (OuterVolumeSpecName: "home") pod "b16e79a0-fd3b-49ab-bf2c-63b8742b40b6" (UID: "b16e79a0-fd3b-49ab-bf2c-63b8742b40b6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:58:41.889935 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.889882    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b16e79a0-fd3b-49ab-bf2c-63b8742b40b6" (UID: "b16e79a0-fd3b-49ab-bf2c-63b8742b40b6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:58:41.890030 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.889962    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-model-cache" (OuterVolumeSpecName: "model-cache") pod "b16e79a0-fd3b-49ab-bf2c-63b8742b40b6" (UID: "b16e79a0-fd3b-49ab-bf2c-63b8742b40b6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:58:41.890069 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.890049    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-model-cache\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:41.890131 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.890069    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-home\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:41.890131 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.890083    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-tmp-dir\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:41.891647 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.891624    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-dshm" (OuterVolumeSpecName: "dshm") pod "b16e79a0-fd3b-49ab-bf2c-63b8742b40b6" (UID: "b16e79a0-fd3b-49ab-bf2c-63b8742b40b6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:58:41.891747 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.891730    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-kube-api-access-wrtdx" (OuterVolumeSpecName: "kube-api-access-wrtdx") pod "b16e79a0-fd3b-49ab-bf2c-63b8742b40b6" (UID: "b16e79a0-fd3b-49ab-bf2c-63b8742b40b6"). InnerVolumeSpecName "kube-api-access-wrtdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:58:41.892149 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.892126    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b16e79a0-fd3b-49ab-bf2c-63b8742b40b6" (UID: "b16e79a0-fd3b-49ab-bf2c-63b8742b40b6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 13:58:41.990669 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.990637    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrtdx\" (UniqueName: \"kubernetes.io/projected/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-kube-api-access-wrtdx\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:41.990669 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.990665    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-dshm\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:41.990669 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:41.990674    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6-tls-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:42.379456 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.379416    2557 generic.go:356] "Generic (PLEG): container finished" podID="b16e79a0-fd3b-49ab-bf2c-63b8742b40b6" containerID="a65eb2672e976871dad5ada2a9a5003c118f21f8011fd8da2d1db7395cea2bc0" exitCode=0
Jul 30 13:58:42.379683 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.379464    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"
Jul 30 13:58:42.379683 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.379509    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs" event={"ID":"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6","Type":"ContainerDied","Data":"a65eb2672e976871dad5ada2a9a5003c118f21f8011fd8da2d1db7395cea2bc0"}
Jul 30 13:58:42.379683 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.379556    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs" event={"ID":"b16e79a0-fd3b-49ab-bf2c-63b8742b40b6","Type":"ContainerDied","Data":"2e21d63488c8d68e15c558a27c3245b011a3453062005e5f0f9d374170a3d74a"}
Jul 30 13:58:42.379683 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.379579    2557 scope.go:117] "RemoveContainer" containerID="a65eb2672e976871dad5ada2a9a5003c118f21f8011fd8da2d1db7395cea2bc0"
Jul 30 13:58:42.381061 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.381033    2557 generic.go:356] "Generic (PLEG): container finished" podID="ddd6202d-1741-4084-afa9-046fb6c96d72" containerID="51663faf5cf1e7a5dcf10c503dde11c126445f947b8f3a9d917834902dcc27d5" exitCode=0
Jul 30 13:58:42.381180 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.381067    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140093674254384" event={"ID":"ddd6202d-1741-4084-afa9-046fb6c96d72","Type":"ContainerDied","Data":"51663faf5cf1e7a5dcf10c503dde11c126445f947b8f3a9d917834902dcc27d5"}
Jul 30 13:58:42.388928 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.388913    2557 scope.go:117] "RemoveContainer" containerID="a65eb2672e976871dad5ada2a9a5003c118f21f8011fd8da2d1db7395cea2bc0"
Jul 30 13:58:42.389157 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:58:42.389139    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a65eb2672e976871dad5ada2a9a5003c118f21f8011fd8da2d1db7395cea2bc0\": container with ID starting with a65eb2672e976871dad5ada2a9a5003c118f21f8011fd8da2d1db7395cea2bc0 not found: ID does not exist" containerID="a65eb2672e976871dad5ada2a9a5003c118f21f8011fd8da2d1db7395cea2bc0"
Jul 30 13:58:42.389200 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.389165    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a65eb2672e976871dad5ada2a9a5003c118f21f8011fd8da2d1db7395cea2bc0"} err="failed to get container status \"a65eb2672e976871dad5ada2a9a5003c118f21f8011fd8da2d1db7395cea2bc0\": rpc error: code = NotFound desc = could not find container \"a65eb2672e976871dad5ada2a9a5003c118f21f8011fd8da2d1db7395cea2bc0\": container with ID starting with a65eb2672e976871dad5ada2a9a5003c118f21f8011fd8da2d1db7395cea2bc0 not found: ID does not exist"
Jul 30 13:58:42.412319 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.412289    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"]
Jul 30 13:58:42.416848 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.416825    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f4dc0a/rate-test-model-f4dc0a-kserve-7b4b4fbb9f-sq5zs"]
Jul 30 13:58:42.548897 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.548856    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584" podUID="b969d9a4-8d43-4257-baee-a1259728a79e" containerName="main" containerID="cri-o://445e0fc1596dc9f251f945e77ff312e5bc330dcb6367d8037670491da8164c33" gracePeriod=45
Jul 30 13:58:42.802148 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.802107    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:58:42.899012 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.898981    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-home\") pod \"b969d9a4-8d43-4257-baee-a1259728a79e\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") "
Jul 30 13:58:42.899159 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.899040    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-tmp-dir\") pod \"b969d9a4-8d43-4257-baee-a1259728a79e\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") "
Jul 30 13:58:42.899159 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.899067    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nc2w5\" (UniqueName: \"kubernetes.io/projected/b969d9a4-8d43-4257-baee-a1259728a79e-kube-api-access-nc2w5\") pod \"b969d9a4-8d43-4257-baee-a1259728a79e\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") "
Jul 30 13:58:42.899159 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.899091    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b969d9a4-8d43-4257-baee-a1259728a79e-tls-certs\") pod \"b969d9a4-8d43-4257-baee-a1259728a79e\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") "
Jul 30 13:58:42.899159 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.899110    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-model-cache\") pod \"b969d9a4-8d43-4257-baee-a1259728a79e\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") "
Jul 30 13:58:42.899159 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.899136    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-dshm\") pod \"b969d9a4-8d43-4257-baee-a1259728a79e\" (UID: \"b969d9a4-8d43-4257-baee-a1259728a79e\") "
Jul 30 13:58:42.899371 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.899234    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-home" (OuterVolumeSpecName: "home") pod "b969d9a4-8d43-4257-baee-a1259728a79e" (UID: "b969d9a4-8d43-4257-baee-a1259728a79e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:58:42.899408 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.899380    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-model-cache" (OuterVolumeSpecName: "model-cache") pod "b969d9a4-8d43-4257-baee-a1259728a79e" (UID: "b969d9a4-8d43-4257-baee-a1259728a79e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:58:42.899452 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.899397    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b969d9a4-8d43-4257-baee-a1259728a79e" (UID: "b969d9a4-8d43-4257-baee-a1259728a79e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:58:42.899452 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.899424    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-home\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:42.901340 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.901315    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-dshm" (OuterVolumeSpecName: "dshm") pod "b969d9a4-8d43-4257-baee-a1259728a79e" (UID: "b969d9a4-8d43-4257-baee-a1259728a79e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 13:58:42.901340 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.901324    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b969d9a4-8d43-4257-baee-a1259728a79e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b969d9a4-8d43-4257-baee-a1259728a79e" (UID: "b969d9a4-8d43-4257-baee-a1259728a79e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 13:58:42.901449 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:42.901358    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b969d9a4-8d43-4257-baee-a1259728a79e-kube-api-access-nc2w5" (OuterVolumeSpecName: "kube-api-access-nc2w5") pod "b969d9a4-8d43-4257-baee-a1259728a79e" (UID: "b969d9a4-8d43-4257-baee-a1259728a79e"). InnerVolumeSpecName "kube-api-access-nc2w5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:58:43.000966 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.000879    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-tmp-dir\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:43.000966 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.000907    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc2w5\" (UniqueName: \"kubernetes.io/projected/b969d9a4-8d43-4257-baee-a1259728a79e-kube-api-access-nc2w5\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:43.000966 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.000918    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b969d9a4-8d43-4257-baee-a1259728a79e-tls-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:43.000966 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.000926    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-model-cache\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:43.000966 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.000933    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b969d9a4-8d43-4257-baee-a1259728a79e-dshm\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:43.183431 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.183398    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b16e79a0-fd3b-49ab-bf2c-63b8742b40b6" path="/var/lib/kubelet/pods/b16e79a0-fd3b-49ab-bf2c-63b8742b40b6/volumes"
Jul 30 13:58:43.387646 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.387538    2557 generic.go:356] "Generic (PLEG): container finished" podID="b969d9a4-8d43-4257-baee-a1259728a79e" containerID="445e0fc1596dc9f251f945e77ff312e5bc330dcb6367d8037670491da8164c33" exitCode=0
Jul 30 13:58:43.387646 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.387583    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"
Jul 30 13:58:43.387646 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.387636    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584" event={"ID":"b969d9a4-8d43-4257-baee-a1259728a79e","Type":"ContainerDied","Data":"445e0fc1596dc9f251f945e77ff312e5bc330dcb6367d8037670491da8164c33"}
Jul 30 13:58:43.387909 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.387685    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584" event={"ID":"b969d9a4-8d43-4257-baee-a1259728a79e","Type":"ContainerDied","Data":"5c0b0f4931576ee424a2084bdadf9a87e92d6c8c82e300e37088a26da5423cda"}
Jul 30 13:58:43.387909 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.387704    2557 scope.go:117] "RemoveContainer" containerID="445e0fc1596dc9f251f945e77ff312e5bc330dcb6367d8037670491da8164c33"
Jul 30 13:58:43.396953 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.396937    2557 scope.go:117] "RemoveContainer" containerID="445e0fc1596dc9f251f945e77ff312e5bc330dcb6367d8037670491da8164c33"
Jul 30 13:58:43.397205 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:58:43.397189    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"445e0fc1596dc9f251f945e77ff312e5bc330dcb6367d8037670491da8164c33\": container with ID starting with 445e0fc1596dc9f251f945e77ff312e5bc330dcb6367d8037670491da8164c33 not found: ID does not exist" containerID="445e0fc1596dc9f251f945e77ff312e5bc330dcb6367d8037670491da8164c33"
Jul 30 13:58:43.397253 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.397213    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"445e0fc1596dc9f251f945e77ff312e5bc330dcb6367d8037670491da8164c33"} err="failed to get container status \"445e0fc1596dc9f251f945e77ff312e5bc330dcb6367d8037670491da8164c33\": rpc error: code = NotFound desc = could not find container \"445e0fc1596dc9f251f945e77ff312e5bc330dcb6367d8037670491da8164c33\": container with ID starting with 445e0fc1596dc9f251f945e77ff312e5bc330dcb6367d8037670491da8164c33 not found: ID does not exist"
Jul 30 13:58:43.405249 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.405227    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"]
Jul 30 13:58:43.410100 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.410078    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e945a1/rate-test-model-e945a1-kserve-86ddf757f-89584"]
Jul 30 13:58:43.543588 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.543564    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674254384"
Jul 30 13:58:43.567448 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.567416    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7483-140093674254384_ddd6202d-1741-4084-afa9-046fb6c96d72/test-curl-7483-140093674254384/0.log"
Jul 30 13:58:43.595835 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.595803    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7483-140093674254384"]
Jul 30 13:58:43.598231 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.598207    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7483-140093674254384"]
Jul 30 13:58:43.612897 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.612872    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc657\" (UniqueName: \"kubernetes.io/projected/ddd6202d-1741-4084-afa9-046fb6c96d72-kube-api-access-vc657\") pod \"ddd6202d-1741-4084-afa9-046fb6c96d72\" (UID: \"ddd6202d-1741-4084-afa9-046fb6c96d72\") "
Jul 30 13:58:43.614846 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.614823    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddd6202d-1741-4084-afa9-046fb6c96d72-kube-api-access-vc657" (OuterVolumeSpecName: "kube-api-access-vc657") pod "ddd6202d-1741-4084-afa9-046fb6c96d72" (UID: "ddd6202d-1741-4084-afa9-046fb6c96d72"). InnerVolumeSpecName "kube-api-access-vc657". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:58:43.713523 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.713495    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc657\" (UniqueName: \"kubernetes.io/projected/ddd6202d-1741-4084-afa9-046fb6c96d72-kube-api-access-vc657\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:43.720239 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.720214    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7483-140093674254384"]
Jul 30 13:58:43.720566 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.720555    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b16e79a0-fd3b-49ab-bf2c-63b8742b40b6" containerName="main"
Jul 30 13:58:43.720625 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.720568    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b16e79a0-fd3b-49ab-bf2c-63b8742b40b6" containerName="main"
Jul 30 13:58:43.720625 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.720584    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd6202d-1741-4084-afa9-046fb6c96d72" containerName="test-curl-7483-140093674254384"
Jul 30 13:58:43.720625 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.720590    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd6202d-1741-4084-afa9-046fb6c96d72" containerName="test-curl-7483-140093674254384"
Jul 30 13:58:43.720721 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.720626    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b969d9a4-8d43-4257-baee-a1259728a79e" containerName="main"
Jul 30 13:58:43.720721 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.720634    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b969d9a4-8d43-4257-baee-a1259728a79e" containerName="main"
Jul 30 13:58:43.720721 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.720696    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b16e79a0-fd3b-49ab-bf2c-63b8742b40b6" containerName="main"
Jul 30 13:58:43.720721 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.720706    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b969d9a4-8d43-4257-baee-a1259728a79e" containerName="main"
Jul 30 13:58:43.720721 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.720713    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddd6202d-1741-4084-afa9-046fb6c96d72" containerName="test-curl-7483-140093674254384"
Jul 30 13:58:43.723637 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.723620    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674254384"
Jul 30 13:58:43.730496 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.730383    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140093674254384"]
Jul 30 13:58:43.814267 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.814232    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsqrn\" (UniqueName: \"kubernetes.io/projected/c6cc4fa5-233b-4b75-b7f9-bdb84b279937-kube-api-access-hsqrn\") pod \"test-curl-7483-140093674254384\" (UID: \"c6cc4fa5-233b-4b75-b7f9-bdb84b279937\") " pod="opendatahub/test-curl-7483-140093674254384"
Jul 30 13:58:43.915441 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.915404    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsqrn\" (UniqueName: \"kubernetes.io/projected/c6cc4fa5-233b-4b75-b7f9-bdb84b279937-kube-api-access-hsqrn\") pod \"test-curl-7483-140093674254384\" (UID: \"c6cc4fa5-233b-4b75-b7f9-bdb84b279937\") " pod="opendatahub/test-curl-7483-140093674254384"
Jul 30 13:58:43.925081 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:43.925045    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsqrn\" (UniqueName: \"kubernetes.io/projected/c6cc4fa5-233b-4b75-b7f9-bdb84b279937-kube-api-access-hsqrn\") pod \"test-curl-7483-140093674254384\" (UID: \"c6cc4fa5-233b-4b75-b7f9-bdb84b279937\") " pod="opendatahub/test-curl-7483-140093674254384"
Jul 30 13:58:44.033504 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:44.033423    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674254384"
Jul 30 13:58:44.158956 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:44.158930    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7483-140093674254384"]
Jul 30 13:58:44.161499 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:58:44.161475    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6cc4fa5_233b_4b75_b7f9_bdb84b279937.slice/crio-fe67ba62d5690a8f079cf0f82147b033603991900b2ee0dcaf350828d3b15af3 WatchSource:0}: Error finding container fe67ba62d5690a8f079cf0f82147b033603991900b2ee0dcaf350828d3b15af3: Status 404 returned error can't find the container with id fe67ba62d5690a8f079cf0f82147b033603991900b2ee0dcaf350828d3b15af3
Jul 30 13:58:44.395244 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:44.395159    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60063fce7f192d37e32e085cbe5f16561fec850654eb1605c37ef3527682635b"
Jul 30 13:58:44.395244 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:44.395162    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674254384"
Jul 30 13:58:44.396250 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:44.396225    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140093674254384" event={"ID":"c6cc4fa5-233b-4b75-b7f9-bdb84b279937","Type":"ContainerStarted","Data":"fe67ba62d5690a8f079cf0f82147b033603991900b2ee0dcaf350828d3b15af3"}
Jul 30 13:58:44.407781 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:44.407755    2557 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7483-140093674254384" oldPodUID="ddd6202d-1741-4084-afa9-046fb6c96d72" podUID="c6cc4fa5-233b-4b75-b7f9-bdb84b279937"
Jul 30 13:58:45.183763 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:45.183730    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b969d9a4-8d43-4257-baee-a1259728a79e" path="/var/lib/kubelet/pods/b969d9a4-8d43-4257-baee-a1259728a79e/volumes"
Jul 30 13:58:45.184123 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:45.184059    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddd6202d-1741-4084-afa9-046fb6c96d72" path="/var/lib/kubelet/pods/ddd6202d-1741-4084-afa9-046fb6c96d72/volumes"
Jul 30 13:58:45.403461 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:45.403424    2557 generic.go:356] "Generic (PLEG): container finished" podID="c6cc4fa5-233b-4b75-b7f9-bdb84b279937" containerID="3e84ac19022c9bc9dba0825245442b39d331178d6c5881822e31f51989e1139c" exitCode=0
Jul 30 13:58:45.403630 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:45.403502    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7483-140093674254384" event={"ID":"c6cc4fa5-233b-4b75-b7f9-bdb84b279937","Type":"ContainerDied","Data":"3e84ac19022c9bc9dba0825245442b39d331178d6c5881822e31f51989e1139c"}
Jul 30 13:58:46.537231 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:46.537207    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674254384"
Jul 30 13:58:46.567474 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:46.567399    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7483-140093674254384_c6cc4fa5-233b-4b75-b7f9-bdb84b279937/test-curl-7483-140093674254384/0.log"
Jul 30 13:58:46.608186 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:46.608156    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7483-140093674254384"]
Jul 30 13:58:46.627149 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:46.627123    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7483-140093674254384"]
Jul 30 13:58:46.634639 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:46.634615    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsqrn\" (UniqueName: \"kubernetes.io/projected/c6cc4fa5-233b-4b75-b7f9-bdb84b279937-kube-api-access-hsqrn\") pod \"c6cc4fa5-233b-4b75-b7f9-bdb84b279937\" (UID: \"c6cc4fa5-233b-4b75-b7f9-bdb84b279937\") "
Jul 30 13:58:46.636635 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:46.636606    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6cc4fa5-233b-4b75-b7f9-bdb84b279937-kube-api-access-hsqrn" (OuterVolumeSpecName: "kube-api-access-hsqrn") pod "c6cc4fa5-233b-4b75-b7f9-bdb84b279937" (UID: "c6cc4fa5-233b-4b75-b7f9-bdb84b279937"). InnerVolumeSpecName "kube-api-access-hsqrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 13:58:46.735304 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:46.735220    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsqrn\" (UniqueName: \"kubernetes.io/projected/c6cc4fa5-233b-4b75-b7f9-bdb84b279937-kube-api-access-hsqrn\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 13:58:47.184823 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.184791    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6cc4fa5-233b-4b75-b7f9-bdb84b279937" path="/var/lib/kubelet/pods/c6cc4fa5-233b-4b75-b7f9-bdb84b279937/volumes"
Jul 30 13:58:47.402874 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.402837    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"]
Jul 30 13:58:47.403198 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.403185    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6cc4fa5-233b-4b75-b7f9-bdb84b279937" containerName="test-curl-7483-140093674254384"
Jul 30 13:58:47.403275 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.403199    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6cc4fa5-233b-4b75-b7f9-bdb84b279937" containerName="test-curl-7483-140093674254384"
Jul 30 13:58:47.403329 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.403288    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6cc4fa5-233b-4b75-b7f9-bdb84b279937" containerName="test-curl-7483-140093674254384"
Jul 30 13:58:47.406345 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.406326    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.409222 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.409192    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-cd11f9-openshift-default-dockercfg-bdbgx\""
Jul 30 13:58:47.416177 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.416153    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7483-140093674254384"
Jul 30 13:58:47.416266 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.416182    2557 scope.go:117] "RemoveContainer" containerID="3e84ac19022c9bc9dba0825245442b39d331178d6c5881822e31f51989e1139c"
Jul 30 13:58:47.421282 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.421257    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"]
Jul 30 13:58:47.441456 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.441376    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-envoy\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.441456 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.441412    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-data\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.441456 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.441435    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c08da357-1adf-4e23-a937-bf3c971dcb47-istiod-ca-cert\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.441456 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.441450    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-credential-socket\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.441752 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.441504    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-podinfo\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.441752 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.441531    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-token\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.441752 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.441573    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-workload-certs\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.441752 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.441641    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ztbw\" (UniqueName: \"kubernetes.io/projected/c08da357-1adf-4e23-a937-bf3c971dcb47-kube-api-access-7ztbw\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.441752 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.441677    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-workload-socket\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.543028 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.542994    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-token\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.543417 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.543050    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-workload-certs\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.543417 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.543168    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ztbw\" (UniqueName: \"kubernetes.io/projected/c08da357-1adf-4e23-a937-bf3c971dcb47-kube-api-access-7ztbw\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.543417 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.543221    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-workload-socket\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.543417 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.543271    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-envoy\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.543417 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.543306    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-data\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.543726 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.543432    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c08da357-1adf-4e23-a937-bf3c971dcb47-istiod-ca-cert\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.543726 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.543480    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-credential-socket\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.543726 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.543524    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-podinfo\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.543726 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.543561    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-workload-certs\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.543726 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.543644    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-data\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.543726 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.543666    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-workload-socket\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.544044 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.543841    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-credential-socket\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.544421 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.544398    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c08da357-1adf-4e23-a937-bf3c971dcb47-istiod-ca-cert\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.545668 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.545648    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-envoy\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.545925 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.545905    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-podinfo\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.551911 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.551889    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-token\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.552012 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.551982    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ztbw\" (UniqueName: \"kubernetes.io/projected/c08da357-1adf-4e23-a937-bf3c971dcb47-kube-api-access-7ztbw\") pod \"e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") " pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.717632 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.717526    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:47.857200 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.857174    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"]
Jul 30 13:58:47.858971 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:58:47.858942    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc08da357_1adf_4e23_a937_bf3c971dcb47.slice/crio-826fd389445127d0580aa148f19bf2f926292e3cb27582ca071de7e851de8fcc WatchSource:0}: Error finding container 826fd389445127d0580aa148f19bf2f926292e3cb27582ca071de7e851de8fcc: Status 404 returned error can't find the container with id 826fd389445127d0580aa148f19bf2f926292e3cb27582ca071de7e851de8fcc
Jul 30 13:58:47.861328 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.861296    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:58:47.861426 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.861360    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:58:47.861426 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:47.861389    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:58:48.423693 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:48.423657    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h" event={"ID":"c08da357-1adf-4e23-a937-bf3c971dcb47","Type":"ContainerStarted","Data":"cb71fae6013ac559d583a517ee5cfb54a616f51b5b3ca72f55d39506c77481dc"}
Jul 30 13:58:48.423873 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:48.423723    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h" event={"ID":"c08da357-1adf-4e23-a937-bf3c971dcb47","Type":"ContainerStarted","Data":"826fd389445127d0580aa148f19bf2f926292e3cb27582ca071de7e851de8fcc"}
Jul 30 13:58:48.448676 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:48.448625    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h" podStartSLOduration=1.448610239 podStartE2EDuration="1.448610239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:58:48.44736132 +0000 UTC m=+3551.719825716" watchObservedRunningTime="2026-07-30 13:58:48.448610239 +0000 UTC m=+3551.721074627"
Jul 30 13:58:48.718570 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:48.718477    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:48.723428 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:48.723405    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:49.429693 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:49.429660    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:49.430667 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:49.430642    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:58:51.455265 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:51.455232    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"]
Jul 30 13:58:51.458690 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:51.458669    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 13:58:51.461565 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:51.461542    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-cd11f9-serving-cert\""
Jul 30 13:58:51.471066 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:51.471039    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"]
Jul 30 13:58:51.480350 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:51.480320    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx\" (UID: \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 13:58:51.480524 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:51.480504    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmwcr\" (UniqueName: \"kubernetes.io/projected/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-kube-api-access-kmwcr\") pod \"maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx\" (UID: \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 13:58:51.480654 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:51.480553    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx\" (UID: \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 13:58:51.581148 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:51.581117    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmwcr\" (UniqueName: \"kubernetes.io/projected/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-kube-api-access-kmwcr\") pod \"maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx\" (UID: \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 13:58:51.581356 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:51.581159    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx\" (UID: \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 13:58:51.581356 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:51.581204    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx\" (UID: \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 13:58:51.581356 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:58:51.581318    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-serving-cert: secret "maas-api-e2e-ipp-a-cd11f9-serving-cert" not found
Jul 30 13:58:51.581551 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:58:51.581386    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-maas-api-tls podName:b19ecb58-ff4e-46d3-a0d7-a034ad2409c9 nodeName:}" failed. No retries permitted until 2026-07-30 13:58:52.081369717 +0000 UTC m=+3555.353834092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-maas-api-tls") pod "maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx" (UID: "b19ecb58-ff4e-46d3-a0d7-a034ad2409c9") : secret "maas-api-e2e-ipp-a-cd11f9-serving-cert" not found
Jul 30 13:58:51.581848 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:51.581830    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx\" (UID: \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 13:58:51.591626 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:51.591592    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmwcr\" (UniqueName: \"kubernetes.io/projected/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-kube-api-access-kmwcr\") pod \"maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx\" (UID: \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 13:58:52.085615 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:52.085558    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx\" (UID: \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 13:58:52.087917 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:52.087891    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx\" (UID: \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 13:58:52.371519 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:52.371435    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 13:58:52.506980 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:52.506950    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"]
Jul 30 13:58:52.509256 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:58:52.509234    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb19ecb58_ff4e_46d3_a0d7_a034ad2409c9.slice/crio-e194101e80eb0abf2b321b881b256e6eea8a1c206f039906926ae4b9b6f45a0f WatchSource:0}: Error finding container e194101e80eb0abf2b321b881b256e6eea8a1c206f039906926ae4b9b6f45a0f: Status 404 returned error can't find the container with id e194101e80eb0abf2b321b881b256e6eea8a1c206f039906926ae4b9b6f45a0f
Jul 30 13:58:53.451578 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:53.451541    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx" event={"ID":"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9","Type":"ContainerStarted","Data":"694b983226fb64dab80e4a33f918badb4d5dc218996d33febd74a5dd26527927"}
Jul 30 13:58:53.451765 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:53.451613    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx" event={"ID":"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9","Type":"ContainerStarted","Data":"e194101e80eb0abf2b321b881b256e6eea8a1c206f039906926ae4b9b6f45a0f"}
Jul 30 13:58:53.451765 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:53.451638    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 13:58:53.472894 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:53.471477    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx" podStartSLOduration=1.9189901740000002 podStartE2EDuration="2.471460366s" totalImagesPullingTime="552.470192ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:58:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:58:53.471167449 +0000 UTC m=+3556.743631858" watchObservedRunningTime="2026-07-30 13:58:53.471460366 +0000 UTC m=+3556.743924763"
Jul 30 13:58:59.462256 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:59.462223    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 13:58:59.529110 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:58:59.529078    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:59:05.471803 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:05.471755    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 13:59:05.472219 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:05.471836    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h" podUID="c08da357-1adf-4e23-a937-bf3c971dcb47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 13:59:07.528932 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:07.528843    2557 generic.go:356] "Generic (PLEG): container finished" podID="c08da357-1adf-4e23-a937-bf3c971dcb47" containerID="cb71fae6013ac559d583a517ee5cfb54a616f51b5b3ca72f55d39506c77481dc" exitCode=0
Jul 30 13:59:07.528932 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:07.528902    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h" event={"ID":"c08da357-1adf-4e23-a937-bf3c971dcb47","Type":"ContainerDied","Data":"cb71fae6013ac559d583a517ee5cfb54a616f51b5b3ca72f55d39506c77481dc"}
Jul 30 13:59:07.529385 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:07.529350    2557 scope.go:117] "RemoveContainer" containerID="cb71fae6013ac559d583a517ee5cfb54a616f51b5b3ca72f55d39506c77481dc"
Jul 30 13:59:07.530431 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:07.530406    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:59:07.530495 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:07.530476    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:59:07.530530 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:07.530510    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 13:59:08.535178 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:08.535149    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h" event={"ID":"c08da357-1adf-4e23-a937-bf3c971dcb47","Type":"ContainerStarted","Data":"c37f6e02f0e6c488f41b09df85c82989ebf572894a55e767e9c57d1f1e914119"}
Jul 30 13:59:08.717742 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:08.717703    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:59:08.718015 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:08.717994    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Jul 30 13:59:08.718067 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:08.718040    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h" podUID="c08da357-1adf-4e23-a937-bf3c971dcb47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Jul 30 13:59:09.718223 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:09.718191    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Jul 30 13:59:09.718661 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:09.718247    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h" podUID="c08da357-1adf-4e23-a937-bf3c971dcb47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Jul 30 13:59:10.718515 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:10.718480    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Jul 30 13:59:10.718938 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:10.718545    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h" podUID="c08da357-1adf-4e23-a937-bf3c971dcb47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Jul 30 13:59:11.718650 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:11.718590    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Jul 30 13:59:11.719052 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:11.718672    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h" podUID="c08da357-1adf-4e23-a937-bf3c971dcb47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Jul 30 13:59:12.722024 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:12.721988    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:59:12.722438 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:12.722364    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:59:12.723061 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:12.723045    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 13:59:20.600649 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:20.600611    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"]
Jul 30 13:59:20.605396 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:20.605376    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 13:59:20.608369 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:20.608337    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-2a5bef-serving-cert\""
Jul 30 13:59:20.614987 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:20.614956    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"]
Jul 30 13:59:20.677245 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:20.677214    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b346b520-732e-413f-beb2-27fd874cc12c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n\" (UID: \"b346b520-732e-413f-beb2-27fd874cc12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 13:59:20.677462 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:20.677272    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b346b520-732e-413f-beb2-27fd874cc12c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n\" (UID: \"b346b520-732e-413f-beb2-27fd874cc12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 13:59:20.677462 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:20.677409    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsfmd\" (UniqueName: \"kubernetes.io/projected/b346b520-732e-413f-beb2-27fd874cc12c-kube-api-access-wsfmd\") pod \"maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n\" (UID: \"b346b520-732e-413f-beb2-27fd874cc12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 13:59:20.778438 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:20.778401    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsfmd\" (UniqueName: \"kubernetes.io/projected/b346b520-732e-413f-beb2-27fd874cc12c-kube-api-access-wsfmd\") pod \"maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n\" (UID: \"b346b520-732e-413f-beb2-27fd874cc12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 13:59:20.778676 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:20.778447    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b346b520-732e-413f-beb2-27fd874cc12c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n\" (UID: \"b346b520-732e-413f-beb2-27fd874cc12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 13:59:20.778676 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:20.778497    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b346b520-732e-413f-beb2-27fd874cc12c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n\" (UID: \"b346b520-732e-413f-beb2-27fd874cc12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 13:59:20.778828 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:59:20.778676    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-serving-cert: secret "maas-api-e2e-ipp-b-2a5bef-serving-cert" not found
Jul 30 13:59:20.778828 ip-10-0-143-33 kubenswrapper[2557]: E0730 13:59:20.778756    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b346b520-732e-413f-beb2-27fd874cc12c-maas-api-tls podName:b346b520-732e-413f-beb2-27fd874cc12c nodeName:}" failed. No retries permitted until 2026-07-30 13:59:21.278734948 +0000 UTC m=+3584.551199330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b346b520-732e-413f-beb2-27fd874cc12c-maas-api-tls") pod "maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n" (UID: "b346b520-732e-413f-beb2-27fd874cc12c") : secret "maas-api-e2e-ipp-b-2a5bef-serving-cert" not found
Jul 30 13:59:20.779129 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:20.779111    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b346b520-732e-413f-beb2-27fd874cc12c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n\" (UID: \"b346b520-732e-413f-beb2-27fd874cc12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 13:59:20.788234 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:20.788204    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsfmd\" (UniqueName: \"kubernetes.io/projected/b346b520-732e-413f-beb2-27fd874cc12c-kube-api-access-wsfmd\") pod \"maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n\" (UID: \"b346b520-732e-413f-beb2-27fd874cc12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 13:59:21.282670 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:21.282630    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b346b520-732e-413f-beb2-27fd874cc12c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n\" (UID: \"b346b520-732e-413f-beb2-27fd874cc12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 13:59:21.284986 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:21.284959    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b346b520-732e-413f-beb2-27fd874cc12c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n\" (UID: \"b346b520-732e-413f-beb2-27fd874cc12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 13:59:21.519398 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:21.519359    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 13:59:21.657228 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:21.657200    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"]
Jul 30 13:59:21.659656 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:59:21.659625    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb346b520_732e_413f_beb2_27fd874cc12c.slice/crio-fc80c838210b065b1207bcbc562e0a247479784ef99a1a935b50ced0221b7b4f WatchSource:0}: Error finding container fc80c838210b065b1207bcbc562e0a247479784ef99a1a935b50ced0221b7b4f: Status 404 returned error can't find the container with id fc80c838210b065b1207bcbc562e0a247479784ef99a1a935b50ced0221b7b4f
Jul 30 13:59:22.608292 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:22.608256    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n" event={"ID":"b346b520-732e-413f-beb2-27fd874cc12c","Type":"ContainerStarted","Data":"3d73733abe46e63d42693e5748b4167173c31c7289888bb02c39deafcc4004d4"}
Jul 30 13:59:22.608545 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:22.608335    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n" event={"ID":"b346b520-732e-413f-beb2-27fd874cc12c","Type":"ContainerStarted","Data":"fc80c838210b065b1207bcbc562e0a247479784ef99a1a935b50ced0221b7b4f"}
Jul 30 13:59:22.608545 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:22.608369    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 13:59:22.626504 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:22.626455    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n" podStartSLOduration=2.303137755 podStartE2EDuration="2.626440724s" totalImagesPullingTime="323.302969ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:59:22.625245888 +0000 UTC m=+3585.897710305" watchObservedRunningTime="2026-07-30 13:59:22.626440724 +0000 UTC m=+3585.898905182"
Jul 30 13:59:28.619137 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:28.619105    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 13:59:29.142285 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:29.142244    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 13:59:51.337677 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.337648    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"]
Jul 30 13:59:51.342911 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.342891    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.345979 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.345956    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-cd11f9\"/\"openshift-service-ca.crt\""
Jul 30 13:59:51.346208 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.346193    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-cd11f9\"/\"default-dockercfg-4szdf\""
Jul 30 13:59:51.347404 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.347384    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-cd11f9\"/\"kube-root-ca.crt\""
Jul 30 13:59:51.347480 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.347386    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-cd11f9\"/\"ipp-route-cd11f9-kserve-self-signed-certs\""
Jul 30 13:59:51.353356 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.353332    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"]
Jul 30 13:59:51.470117 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.470086    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-dshm\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.470315 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.470126    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-model-cache\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.470315 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.470144    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-tmp-dir\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.470315 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.470239    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhwnc\" (UniqueName: \"kubernetes.io/projected/59c68172-6fea-4c22-9514-5868f8828a7f-kube-api-access-dhwnc\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.470461 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.470356    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-home\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.470461 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.470402    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59c68172-6fea-4c22-9514-5868f8828a7f-tls-certs\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.570914 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.570880    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-home\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.571127 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.570928    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59c68172-6fea-4c22-9514-5868f8828a7f-tls-certs\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.571127 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.571065    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-dshm\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.571245 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.571128    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-model-cache\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.571245 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.571158    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-tmp-dir\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.571245 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.571231    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhwnc\" (UniqueName: \"kubernetes.io/projected/59c68172-6fea-4c22-9514-5868f8828a7f-kube-api-access-dhwnc\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.571406 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.571343    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-home\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.571630 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.571609    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-model-cache\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.571724 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.571691    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-tmp-dir\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.573491 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.573469    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59c68172-6fea-4c22-9514-5868f8828a7f-tls-certs\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.573697 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.573680    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-dshm\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.580492 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.580467    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhwnc\" (UniqueName: \"kubernetes.io/projected/59c68172-6fea-4c22-9514-5868f8828a7f-kube-api-access-dhwnc\") pod \"ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") " pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.654902 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.654825    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 13:59:51.786322 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:51.786297    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"]
Jul 30 13:59:51.788290 ip-10-0-143-33 kubenswrapper[2557]: W0730 13:59:51.788257    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59c68172_6fea_4c22_9514_5868f8828a7f.slice/crio-7b0fe9f92aec04833c912298d28c959a8b6de38f921c3f103dea7139214fa43e WatchSource:0}: Error finding container 7b0fe9f92aec04833c912298d28c959a8b6de38f921c3f103dea7139214fa43e: Status 404 returned error can't find the container with id 7b0fe9f92aec04833c912298d28c959a8b6de38f921c3f103dea7139214fa43e
Jul 30 13:59:52.764247 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:52.764207    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg" event={"ID":"59c68172-6fea-4c22-9514-5868f8828a7f","Type":"ContainerStarted","Data":"9c8db9f7146bfaee3820134cb0f819137e9a93417fde87174bf1cc6df47b26f7"}
Jul 30 13:59:52.764247 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:52.764254    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg" event={"ID":"59c68172-6fea-4c22-9514-5868f8828a7f","Type":"ContainerStarted","Data":"7b0fe9f92aec04833c912298d28c959a8b6de38f921c3f103dea7139214fa43e"}
Jul 30 13:59:52.783474 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:52.783423    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg" podStartSLOduration=1.783408924 podStartE2EDuration="1.783408924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 13:59:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 13:59:52.781419616 +0000 UTC m=+3616.053884013" watchObservedRunningTime="2026-07-30 13:59:52.783408924 +0000 UTC m=+3616.055873320"
Jul 30 13:59:55.539876 ip-10-0-143-33 kubenswrapper[2557]: I0730 13:59:55.539844    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 14:00:01.655665 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:01.655630    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 14:00:01.656065 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:01.655682    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 14:00:01.668652 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:01.668623    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 14:00:01.825410 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:01.825383    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 14:00:04.165560 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:04.165521    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756955-6wf9q"]
Jul 30 14:00:04.170999 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:04.170974    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756955-6wf9q"]
Jul 30 14:00:05.183722 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:05.183686    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68f6d126-b213-4649-95c8-65727d0695cd" path="/var/lib/kubelet/pods/68f6d126-b213-4649-95c8-65727d0695cd/volumes"
Jul 30 14:00:26.644563 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:26.644530    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 14:00:37.646955 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:37.646923    2557 scope.go:117] "RemoveContainer" containerID="47fcd3c4f9424317e726ca9a59730792da61da1480c3e79543e3fdd5bedb0571"
Jul 30 14:00:43.337013 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:43.336974    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"]
Jul 30 14:00:43.341698 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:43.341682    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:43.344817 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:43.344799    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-0e458a-serving-cert\""
Jul 30 14:00:43.352794 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:43.352771    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"]
Jul 30 14:00:43.387917 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:43.387880    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp\" (UID: \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:43.388120 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:43.387938    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp\" (UID: \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:43.388120 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:43.388035    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6j6x\" (UniqueName: \"kubernetes.io/projected/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-kube-api-access-b6j6x\") pod \"maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp\" (UID: \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:43.489109 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:43.489074    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6j6x\" (UniqueName: \"kubernetes.io/projected/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-kube-api-access-b6j6x\") pod \"maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp\" (UID: \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:43.489294 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:43.489198    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp\" (UID: \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:43.489294 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:43.489254    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp\" (UID: \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:43.489376 ip-10-0-143-33 kubenswrapper[2557]: E0730 14:00:43.489360    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-serving-cert: secret "maas-api-e2e-ipp-cleanup-0e458a-serving-cert" not found
Jul 30 14:00:43.489452 ip-10-0-143-33 kubenswrapper[2557]: E0730 14:00:43.489440    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-maas-api-tls podName:95b53ace-c9a9-4e3f-8731-109cd04b1b3a nodeName:}" failed. No retries permitted until 2026-07-30 14:00:43.989420061 +0000 UTC m=+3667.261884435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp" (UID: "95b53ace-c9a9-4e3f-8731-109cd04b1b3a") : secret "maas-api-e2e-ipp-cleanup-0e458a-serving-cert" not found
Jul 30 14:00:43.489890 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:43.489870    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp\" (UID: \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:43.499557 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:43.499527    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6j6x\" (UniqueName: \"kubernetes.io/projected/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-kube-api-access-b6j6x\") pod \"maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp\" (UID: \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:43.995971 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:43.995924    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp\" (UID: \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:43.996160 ip-10-0-143-33 kubenswrapper[2557]: E0730 14:00:43.996078    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-serving-cert: secret "maas-api-e2e-ipp-cleanup-0e458a-serving-cert" not found
Jul 30 14:00:43.996160 ip-10-0-143-33 kubenswrapper[2557]: E0730 14:00:43.996137    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-maas-api-tls podName:95b53ace-c9a9-4e3f-8731-109cd04b1b3a nodeName:}" failed. No retries permitted until 2026-07-30 14:00:44.996123556 +0000 UTC m=+3668.268587935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp" (UID: "95b53ace-c9a9-4e3f-8731-109cd04b1b3a") : secret "maas-api-e2e-ipp-cleanup-0e458a-serving-cert" not found
Jul 30 14:00:45.005836 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:45.005798    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp\" (UID: \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:45.008279 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:45.008256    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp\" (UID: \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:45.151861 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:45.151825    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:45.289241 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:45.289214    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"]
Jul 30 14:00:45.291722 ip-10-0-143-33 kubenswrapper[2557]: W0730 14:00:45.291690    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b53ace_c9a9_4e3f_8731_109cd04b1b3a.slice/crio-3598fe51e4ebc8ce8bae213ddedc3adb5335954bed15d9b586df95c33c904680 WatchSource:0}: Error finding container 3598fe51e4ebc8ce8bae213ddedc3adb5335954bed15d9b586df95c33c904680: Status 404 returned error can't find the container with id 3598fe51e4ebc8ce8bae213ddedc3adb5335954bed15d9b586df95c33c904680
Jul 30 14:00:46.038847 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:46.038753    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp" event={"ID":"95b53ace-c9a9-4e3f-8731-109cd04b1b3a","Type":"ContainerStarted","Data":"abd0c3e84484fc4665f0a122cf5125fe7e5fcabf44c5f8ed4ba5090b9dcf8e69"}
Jul 30 14:00:46.038847 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:46.038836    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp" event={"ID":"95b53ace-c9a9-4e3f-8731-109cd04b1b3a","Type":"ContainerStarted","Data":"3598fe51e4ebc8ce8bae213ddedc3adb5335954bed15d9b586df95c33c904680"}
Jul 30 14:00:46.039261 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:46.038892    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:46.061742 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:46.061680    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp" podStartSLOduration=2.623181922 podStartE2EDuration="3.061659036s" totalImagesPullingTime="438.477114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:00:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:00:46.058361435 +0000 UTC m=+3669.330825869" watchObservedRunningTime="2026-07-30 14:00:46.061659036 +0000 UTC m=+3669.334123434"
Jul 30 14:00:52.050656 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:52.050579    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:00:53.252141 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:00:53.252096    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 14:01:02.958904 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:02.958809    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"]
Jul 30 14:01:02.959501 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:02.959057    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp" podUID="95b53ace-c9a9-4e3f-8731-109cd04b1b3a" containerName="maas-api" containerID="cri-o://abd0c3e84484fc4665f0a122cf5125fe7e5fcabf44c5f8ed4ba5090b9dcf8e69" gracePeriod=30
Jul 30 14:01:03.016250 ip-10-0-143-33 kubenswrapper[2557]: E0730 14:01:03.016182    2557 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b53ace_c9a9_4e3f_8731_109cd04b1b3a.slice/crio-conmon-abd0c3e84484fc4665f0a122cf5125fe7e5fcabf44c5f8ed4ba5090b9dcf8e69.scope\": RecentStats: unable to find data in memory cache]"
Jul 30 14:01:03.128249 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:03.128217    2557 generic.go:356] "Generic (PLEG): container finished" podID="95b53ace-c9a9-4e3f-8731-109cd04b1b3a" containerID="abd0c3e84484fc4665f0a122cf5125fe7e5fcabf44c5f8ed4ba5090b9dcf8e69" exitCode=0
Jul 30 14:01:03.128458 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:03.128300    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp" event={"ID":"95b53ace-c9a9-4e3f-8731-109cd04b1b3a","Type":"ContainerDied","Data":"abd0c3e84484fc4665f0a122cf5125fe7e5fcabf44c5f8ed4ba5090b9dcf8e69"}
Jul 30 14:01:03.223963 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:03.223898    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:01:03.378553 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:03.378523    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-maas-api-tls\") pod \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\" (UID: \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\") "
Jul 30 14:01:03.378736 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:03.378645    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-openshift-service-ca\") pod \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\" (UID: \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\") "
Jul 30 14:01:03.378736 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:03.378725    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6j6x\" (UniqueName: \"kubernetes.io/projected/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-kube-api-access-b6j6x\") pod \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\" (UID: \"95b53ace-c9a9-4e3f-8731-109cd04b1b3a\") "
Jul 30 14:01:03.379055 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:03.379022    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "95b53ace-c9a9-4e3f-8731-109cd04b1b3a" (UID: "95b53ace-c9a9-4e3f-8731-109cd04b1b3a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 14:01:03.380652 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:03.380627    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-kube-api-access-b6j6x" (OuterVolumeSpecName: "kube-api-access-b6j6x") pod "95b53ace-c9a9-4e3f-8731-109cd04b1b3a" (UID: "95b53ace-c9a9-4e3f-8731-109cd04b1b3a"). InnerVolumeSpecName "kube-api-access-b6j6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:01:03.380752 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:03.380662    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "95b53ace-c9a9-4e3f-8731-109cd04b1b3a" (UID: "95b53ace-c9a9-4e3f-8731-109cd04b1b3a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 14:01:03.479624 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:03.479507    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-openshift-service-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:01:03.479624 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:03.479536    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6j6x\" (UniqueName: \"kubernetes.io/projected/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-kube-api-access-b6j6x\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:01:03.479624 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:03.479545    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95b53ace-c9a9-4e3f-8731-109cd04b1b3a-maas-api-tls\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:01:04.134386 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:04.134355    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp" event={"ID":"95b53ace-c9a9-4e3f-8731-109cd04b1b3a","Type":"ContainerDied","Data":"3598fe51e4ebc8ce8bae213ddedc3adb5335954bed15d9b586df95c33c904680"}
Jul 30 14:01:04.134883 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:04.134414    2557 scope.go:117] "RemoveContainer" containerID="abd0c3e84484fc4665f0a122cf5125fe7e5fcabf44c5f8ed4ba5090b9dcf8e69"
Jul 30 14:01:04.134883 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:04.134370    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"
Jul 30 14:01:04.158049 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:04.158014    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"]
Jul 30 14:01:04.164759 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:04.164735    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-0e458a-8597bbbb7c-6j2tp"]
Jul 30 14:01:05.184083 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:05.184048    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95b53ace-c9a9-4e3f-8731-109cd04b1b3a" path="/var/lib/kubelet/pods/95b53ace-c9a9-4e3f-8731-109cd04b1b3a/volumes"
Jul 30 14:01:08.340717 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:08.340686    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 14:01:25.343433 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:25.343396    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"]
Jul 30 14:01:25.343942 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:25.343757    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx" podUID="b19ecb58-ff4e-46d3-a0d7-a034ad2409c9" containerName="maas-api" containerID="cri-o://694b983226fb64dab80e4a33f918badb4d5dc218996d33febd74a5dd26527927" gracePeriod=30
Jul 30 14:01:25.804842 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:25.804817    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 14:01:25.889024 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:25.888940    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmwcr\" (UniqueName: \"kubernetes.io/projected/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-kube-api-access-kmwcr\") pod \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\" (UID: \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\") "
Jul 30 14:01:25.889164 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:25.889030    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-maas-api-tls\") pod \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\" (UID: \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\") "
Jul 30 14:01:25.889164 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:25.889080    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-openshift-service-ca\") pod \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\" (UID: \"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9\") "
Jul 30 14:01:25.889477 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:25.889450    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b19ecb58-ff4e-46d3-a0d7-a034ad2409c9" (UID: "b19ecb58-ff4e-46d3-a0d7-a034ad2409c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 14:01:25.890968 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:25.890936    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b19ecb58-ff4e-46d3-a0d7-a034ad2409c9" (UID: "b19ecb58-ff4e-46d3-a0d7-a034ad2409c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 14:01:25.891065 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:25.891027    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-kube-api-access-kmwcr" (OuterVolumeSpecName: "kube-api-access-kmwcr") pod "b19ecb58-ff4e-46d3-a0d7-a034ad2409c9" (UID: "b19ecb58-ff4e-46d3-a0d7-a034ad2409c9"). InnerVolumeSpecName "kube-api-access-kmwcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:01:25.990535 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:25.990501    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-openshift-service-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:01:25.990535 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:25.990529    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmwcr\" (UniqueName: \"kubernetes.io/projected/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-kube-api-access-kmwcr\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:01:25.990740 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:25.990542    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9-maas-api-tls\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:01:26.249313 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:26.249281    2557 generic.go:356] "Generic (PLEG): container finished" podID="b19ecb58-ff4e-46d3-a0d7-a034ad2409c9" containerID="694b983226fb64dab80e4a33f918badb4d5dc218996d33febd74a5dd26527927" exitCode=0
Jul 30 14:01:26.249509 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:26.249322    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"
Jul 30 14:01:26.249509 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:26.249372    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx" event={"ID":"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9","Type":"ContainerDied","Data":"694b983226fb64dab80e4a33f918badb4d5dc218996d33febd74a5dd26527927"}
Jul 30 14:01:26.249509 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:26.249454    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx" event={"ID":"b19ecb58-ff4e-46d3-a0d7-a034ad2409c9","Type":"ContainerDied","Data":"e194101e80eb0abf2b321b881b256e6eea8a1c206f039906926ae4b9b6f45a0f"}
Jul 30 14:01:26.249509 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:26.249482    2557 scope.go:117] "RemoveContainer" containerID="694b983226fb64dab80e4a33f918badb4d5dc218996d33febd74a5dd26527927"
Jul 30 14:01:26.265536 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:26.265514    2557 scope.go:117] "RemoveContainer" containerID="694b983226fb64dab80e4a33f918badb4d5dc218996d33febd74a5dd26527927"
Jul 30 14:01:26.265849 ip-10-0-143-33 kubenswrapper[2557]: E0730 14:01:26.265827    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"694b983226fb64dab80e4a33f918badb4d5dc218996d33febd74a5dd26527927\": container with ID starting with 694b983226fb64dab80e4a33f918badb4d5dc218996d33febd74a5dd26527927 not found: ID does not exist" containerID="694b983226fb64dab80e4a33f918badb4d5dc218996d33febd74a5dd26527927"
Jul 30 14:01:26.265895 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:26.265859    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"694b983226fb64dab80e4a33f918badb4d5dc218996d33febd74a5dd26527927"} err="failed to get container status \"694b983226fb64dab80e4a33f918badb4d5dc218996d33febd74a5dd26527927\": rpc error: code = NotFound desc = could not find container \"694b983226fb64dab80e4a33f918badb4d5dc218996d33febd74a5dd26527927\": container with ID starting with 694b983226fb64dab80e4a33f918badb4d5dc218996d33febd74a5dd26527927 not found: ID does not exist"
Jul 30 14:01:26.278879 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:26.278854    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"]
Jul 30 14:01:26.282417 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:26.282386    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cd11f9-6d7c974b79-4hzsx"]
Jul 30 14:01:27.185420 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:27.185387    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b19ecb58-ff4e-46d3-a0d7-a034ad2409c9" path="/var/lib/kubelet/pods/b19ecb58-ff4e-46d3-a0d7-a034ad2409c9/volumes"
Jul 30 14:01:28.430966 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:28.430933    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 14:01:31.053491 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:31.053447    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"]
Jul 30 14:01:46.072839 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.072775    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg" podUID="59c68172-6fea-4c22-9514-5868f8828a7f" containerName="main" containerID="cri-o://9c8db9f7146bfaee3820134cb0f819137e9a93417fde87174bf1cc6df47b26f7" gracePeriod=45
Jul 30 14:01:46.316293 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.316269    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 14:01:46.355995 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.355898    2557 generic.go:356] "Generic (PLEG): container finished" podID="59c68172-6fea-4c22-9514-5868f8828a7f" containerID="9c8db9f7146bfaee3820134cb0f819137e9a93417fde87174bf1cc6df47b26f7" exitCode=0
Jul 30 14:01:46.355995 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.355954    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"
Jul 30 14:01:46.355995 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.355964    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg" event={"ID":"59c68172-6fea-4c22-9514-5868f8828a7f","Type":"ContainerDied","Data":"9c8db9f7146bfaee3820134cb0f819137e9a93417fde87174bf1cc6df47b26f7"}
Jul 30 14:01:46.356286 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.356017    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg" event={"ID":"59c68172-6fea-4c22-9514-5868f8828a7f","Type":"ContainerDied","Data":"7b0fe9f92aec04833c912298d28c959a8b6de38f921c3f103dea7139214fa43e"}
Jul 30 14:01:46.356286 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.356040    2557 scope.go:117] "RemoveContainer" containerID="9c8db9f7146bfaee3820134cb0f819137e9a93417fde87174bf1cc6df47b26f7"
Jul 30 14:01:46.365101 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.365083    2557 scope.go:117] "RemoveContainer" containerID="9c8db9f7146bfaee3820134cb0f819137e9a93417fde87174bf1cc6df47b26f7"
Jul 30 14:01:46.365431 ip-10-0-143-33 kubenswrapper[2557]: E0730 14:01:46.365408    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c8db9f7146bfaee3820134cb0f819137e9a93417fde87174bf1cc6df47b26f7\": container with ID starting with 9c8db9f7146bfaee3820134cb0f819137e9a93417fde87174bf1cc6df47b26f7 not found: ID does not exist" containerID="9c8db9f7146bfaee3820134cb0f819137e9a93417fde87174bf1cc6df47b26f7"
Jul 30 14:01:46.365526 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.365445    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c8db9f7146bfaee3820134cb0f819137e9a93417fde87174bf1cc6df47b26f7"} err="failed to get container status \"9c8db9f7146bfaee3820134cb0f819137e9a93417fde87174bf1cc6df47b26f7\": rpc error: code = NotFound desc = could not find container \"9c8db9f7146bfaee3820134cb0f819137e9a93417fde87174bf1cc6df47b26f7\": container with ID starting with 9c8db9f7146bfaee3820134cb0f819137e9a93417fde87174bf1cc6df47b26f7 not found: ID does not exist"
Jul 30 14:01:46.486165 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.486136    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-home\") pod \"59c68172-6fea-4c22-9514-5868f8828a7f\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") "
Jul 30 14:01:46.486372 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.486205    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-dshm\") pod \"59c68172-6fea-4c22-9514-5868f8828a7f\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") "
Jul 30 14:01:46.486372 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.486245    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhwnc\" (UniqueName: \"kubernetes.io/projected/59c68172-6fea-4c22-9514-5868f8828a7f-kube-api-access-dhwnc\") pod \"59c68172-6fea-4c22-9514-5868f8828a7f\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") "
Jul 30 14:01:46.486372 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.486275    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-tmp-dir\") pod \"59c68172-6fea-4c22-9514-5868f8828a7f\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") "
Jul 30 14:01:46.486372 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.486316    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-model-cache\") pod \"59c68172-6fea-4c22-9514-5868f8828a7f\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") "
Jul 30 14:01:46.486577 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.486398    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59c68172-6fea-4c22-9514-5868f8828a7f-tls-certs\") pod \"59c68172-6fea-4c22-9514-5868f8828a7f\" (UID: \"59c68172-6fea-4c22-9514-5868f8828a7f\") "
Jul 30 14:01:46.486577 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.486494    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-home" (OuterVolumeSpecName: "home") pod "59c68172-6fea-4c22-9514-5868f8828a7f" (UID: "59c68172-6fea-4c22-9514-5868f8828a7f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:01:46.486702 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.486638    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-model-cache" (OuterVolumeSpecName: "model-cache") pod "59c68172-6fea-4c22-9514-5868f8828a7f" (UID: "59c68172-6fea-4c22-9514-5868f8828a7f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:01:46.486702 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.486656    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59c68172-6fea-4c22-9514-5868f8828a7f" (UID: "59c68172-6fea-4c22-9514-5868f8828a7f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:01:46.486801 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.486783    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-home\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:01:46.486838 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.486807    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-tmp-dir\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:01:46.486838 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.486820    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-model-cache\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:01:46.488546 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.488496    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59c68172-6fea-4c22-9514-5868f8828a7f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59c68172-6fea-4c22-9514-5868f8828a7f" (UID: "59c68172-6fea-4c22-9514-5868f8828a7f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 14:01:46.488687 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.488638    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-dshm" (OuterVolumeSpecName: "dshm") pod "59c68172-6fea-4c22-9514-5868f8828a7f" (UID: "59c68172-6fea-4c22-9514-5868f8828a7f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:01:46.488687 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.488665    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59c68172-6fea-4c22-9514-5868f8828a7f-kube-api-access-dhwnc" (OuterVolumeSpecName: "kube-api-access-dhwnc") pod "59c68172-6fea-4c22-9514-5868f8828a7f" (UID: "59c68172-6fea-4c22-9514-5868f8828a7f"). InnerVolumeSpecName "kube-api-access-dhwnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:01:46.587575 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.587538    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59c68172-6fea-4c22-9514-5868f8828a7f-tls-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:01:46.587575 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.587568    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59c68172-6fea-4c22-9514-5868f8828a7f-dshm\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:01:46.587575 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.587580    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhwnc\" (UniqueName: \"kubernetes.io/projected/59c68172-6fea-4c22-9514-5868f8828a7f-kube-api-access-dhwnc\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:01:46.680499 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.680468    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"]
Jul 30 14:01:46.683997 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:46.683972    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-cd11f9/ipp-route-cd11f9-kserve-7b4ddb99bc-cfflg"]
Jul 30 14:01:47.185632 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:01:47.185584    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59c68172-6fea-4c22-9514-5868f8828a7f" path="/var/lib/kubelet/pods/59c68172-6fea-4c22-9514-5868f8828a7f/volumes"
Jul 30 14:02:14.764661 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:14.764627    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"]
Jul 30 14:02:14.765282 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:14.764910    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h" podUID="c08da357-1adf-4e23-a937-bf3c971dcb47" containerName="istio-proxy" containerID="cri-o://c37f6e02f0e6c488f41b09df85c82989ebf572894a55e767e9c57d1f1e914119" gracePeriod=30
Jul 30 14:02:20.022344 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.022317    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 14:02:20.100519 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.100439    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-workload-socket\") pod \"c08da357-1adf-4e23-a937-bf3c971dcb47\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") "
Jul 30 14:02:20.100519 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.100488    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ztbw\" (UniqueName: \"kubernetes.io/projected/c08da357-1adf-4e23-a937-bf3c971dcb47-kube-api-access-7ztbw\") pod \"c08da357-1adf-4e23-a937-bf3c971dcb47\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") "
Jul 30 14:02:20.100781 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.100525    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-credential-socket\") pod \"c08da357-1adf-4e23-a937-bf3c971dcb47\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") "
Jul 30 14:02:20.100781 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.100545    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-envoy\") pod \"c08da357-1adf-4e23-a937-bf3c971dcb47\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") "
Jul 30 14:02:20.100781 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.100589    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-data\") pod \"c08da357-1adf-4e23-a937-bf3c971dcb47\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") "
Jul 30 14:02:20.100781 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.100671    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-token\") pod \"c08da357-1adf-4e23-a937-bf3c971dcb47\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") "
Jul 30 14:02:20.100781 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.100701    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-podinfo\") pod \"c08da357-1adf-4e23-a937-bf3c971dcb47\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") "
Jul 30 14:02:20.100781 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.100738    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c08da357-1adf-4e23-a937-bf3c971dcb47-istiod-ca-cert\") pod \"c08da357-1adf-4e23-a937-bf3c971dcb47\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") "
Jul 30 14:02:20.100781 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.100745    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c08da357-1adf-4e23-a937-bf3c971dcb47" (UID: "c08da357-1adf-4e23-a937-bf3c971dcb47"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:02:20.100781 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.100769    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-workload-certs\") pod \"c08da357-1adf-4e23-a937-bf3c971dcb47\" (UID: \"c08da357-1adf-4e23-a937-bf3c971dcb47\") "
Jul 30 14:02:20.101194 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.100920    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c08da357-1adf-4e23-a937-bf3c971dcb47" (UID: "c08da357-1adf-4e23-a937-bf3c971dcb47"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:02:20.101194 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.101125    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-workload-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:20.101194 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.101134    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c08da357-1adf-4e23-a937-bf3c971dcb47" (UID: "c08da357-1adf-4e23-a937-bf3c971dcb47"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:02:20.101194 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.101144    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-credential-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:20.101398 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.101223    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c08da357-1adf-4e23-a937-bf3c971dcb47-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c08da357-1adf-4e23-a937-bf3c971dcb47" (UID: "c08da357-1adf-4e23-a937-bf3c971dcb47"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 14:02:20.101689 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.101660    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-data" (OuterVolumeSpecName: "istio-data") pod "c08da357-1adf-4e23-a937-bf3c971dcb47" (UID: "c08da357-1adf-4e23-a937-bf3c971dcb47"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:02:20.102940 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.102914    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c08da357-1adf-4e23-a937-bf3c971dcb47-kube-api-access-7ztbw" (OuterVolumeSpecName: "kube-api-access-7ztbw") pod "c08da357-1adf-4e23-a937-bf3c971dcb47" (UID: "c08da357-1adf-4e23-a937-bf3c971dcb47"). InnerVolumeSpecName "kube-api-access-7ztbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:02:20.103047 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.102920    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c08da357-1adf-4e23-a937-bf3c971dcb47" (UID: "c08da357-1adf-4e23-a937-bf3c971dcb47"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 14:02:20.103047 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.102937    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c08da357-1adf-4e23-a937-bf3c971dcb47" (UID: "c08da357-1adf-4e23-a937-bf3c971dcb47"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:02:20.103181 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.103164    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-token" (OuterVolumeSpecName: "istio-token") pod "c08da357-1adf-4e23-a937-bf3c971dcb47" (UID: "c08da357-1adf-4e23-a937-bf3c971dcb47"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:02:20.127807 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.127778    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"]
Jul 30 14:02:20.128052 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.128025    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n" podUID="b346b520-732e-413f-beb2-27fd874cc12c" containerName="maas-api" containerID="cri-o://3d73733abe46e63d42693e5748b4167173c31c7289888bb02c39deafcc4004d4" gracePeriod=30
Jul 30 14:02:20.202685 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.202656    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-envoy\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:20.202685 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.202681    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-data\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:20.202685 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.202693    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-token\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:20.202685 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.202705    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c08da357-1adf-4e23-a937-bf3c971dcb47-istio-podinfo\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:20.202685 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.202717    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c08da357-1adf-4e23-a937-bf3c971dcb47-istiod-ca-cert\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:20.203076 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.202726    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c08da357-1adf-4e23-a937-bf3c971dcb47-workload-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:20.203076 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.202734    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ztbw\" (UniqueName: \"kubernetes.io/projected/c08da357-1adf-4e23-a937-bf3c971dcb47-kube-api-access-7ztbw\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:20.377250 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.377223    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 14:02:20.505027 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.504990    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsfmd\" (UniqueName: \"kubernetes.io/projected/b346b520-732e-413f-beb2-27fd874cc12c-kube-api-access-wsfmd\") pod \"b346b520-732e-413f-beb2-27fd874cc12c\" (UID: \"b346b520-732e-413f-beb2-27fd874cc12c\") "
Jul 30 14:02:20.505213 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.505057    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b346b520-732e-413f-beb2-27fd874cc12c-openshift-service-ca\") pod \"b346b520-732e-413f-beb2-27fd874cc12c\" (UID: \"b346b520-732e-413f-beb2-27fd874cc12c\") "
Jul 30 14:02:20.505213 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.505110    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b346b520-732e-413f-beb2-27fd874cc12c-maas-api-tls\") pod \"b346b520-732e-413f-beb2-27fd874cc12c\" (UID: \"b346b520-732e-413f-beb2-27fd874cc12c\") "
Jul 30 14:02:20.505469 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.505441    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b346b520-732e-413f-beb2-27fd874cc12c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b346b520-732e-413f-beb2-27fd874cc12c" (UID: "b346b520-732e-413f-beb2-27fd874cc12c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 14:02:20.507162 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.507141    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b346b520-732e-413f-beb2-27fd874cc12c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b346b520-732e-413f-beb2-27fd874cc12c" (UID: "b346b520-732e-413f-beb2-27fd874cc12c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 14:02:20.507235 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.507191    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b346b520-732e-413f-beb2-27fd874cc12c-kube-api-access-wsfmd" (OuterVolumeSpecName: "kube-api-access-wsfmd") pod "b346b520-732e-413f-beb2-27fd874cc12c" (UID: "b346b520-732e-413f-beb2-27fd874cc12c"). InnerVolumeSpecName "kube-api-access-wsfmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:02:20.525721 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.525694    2557 generic.go:356] "Generic (PLEG): container finished" podID="c08da357-1adf-4e23-a937-bf3c971dcb47" containerID="c37f6e02f0e6c488f41b09df85c82989ebf572894a55e767e9c57d1f1e914119" exitCode=0
Jul 30 14:02:20.525868 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.525746    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"
Jul 30 14:02:20.525868 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.525778    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h" event={"ID":"c08da357-1adf-4e23-a937-bf3c971dcb47","Type":"ContainerDied","Data":"c37f6e02f0e6c488f41b09df85c82989ebf572894a55e767e9c57d1f1e914119"}
Jul 30 14:02:20.525868 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.525840    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h" event={"ID":"c08da357-1adf-4e23-a937-bf3c971dcb47","Type":"ContainerDied","Data":"826fd389445127d0580aa148f19bf2f926292e3cb27582ca071de7e851de8fcc"}
Jul 30 14:02:20.526055 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.525874    2557 scope.go:117] "RemoveContainer" containerID="c37f6e02f0e6c488f41b09df85c82989ebf572894a55e767e9c57d1f1e914119"
Jul 30 14:02:20.527252 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.527224    2557 generic.go:356] "Generic (PLEG): container finished" podID="b346b520-732e-413f-beb2-27fd874cc12c" containerID="3d73733abe46e63d42693e5748b4167173c31c7289888bb02c39deafcc4004d4" exitCode=0
Jul 30 14:02:20.527382 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.527263    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n" event={"ID":"b346b520-732e-413f-beb2-27fd874cc12c","Type":"ContainerDied","Data":"3d73733abe46e63d42693e5748b4167173c31c7289888bb02c39deafcc4004d4"}
Jul 30 14:02:20.527382 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.527290    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"
Jul 30 14:02:20.527382 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.527303    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n" event={"ID":"b346b520-732e-413f-beb2-27fd874cc12c","Type":"ContainerDied","Data":"fc80c838210b065b1207bcbc562e0a247479784ef99a1a935b50ced0221b7b4f"}
Jul 30 14:02:20.537177 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.537155    2557 scope.go:117] "RemoveContainer" containerID="cb71fae6013ac559d583a517ee5cfb54a616f51b5b3ca72f55d39506c77481dc"
Jul 30 14:02:20.545750 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.545709    2557 scope.go:117] "RemoveContainer" containerID="c37f6e02f0e6c488f41b09df85c82989ebf572894a55e767e9c57d1f1e914119"
Jul 30 14:02:20.545973 ip-10-0-143-33 kubenswrapper[2557]: E0730 14:02:20.545955    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c37f6e02f0e6c488f41b09df85c82989ebf572894a55e767e9c57d1f1e914119\": container with ID starting with c37f6e02f0e6c488f41b09df85c82989ebf572894a55e767e9c57d1f1e914119 not found: ID does not exist" containerID="c37f6e02f0e6c488f41b09df85c82989ebf572894a55e767e9c57d1f1e914119"
Jul 30 14:02:20.546026 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.545980    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c37f6e02f0e6c488f41b09df85c82989ebf572894a55e767e9c57d1f1e914119"} err="failed to get container status \"c37f6e02f0e6c488f41b09df85c82989ebf572894a55e767e9c57d1f1e914119\": rpc error: code = NotFound desc = could not find container \"c37f6e02f0e6c488f41b09df85c82989ebf572894a55e767e9c57d1f1e914119\": container with ID starting with c37f6e02f0e6c488f41b09df85c82989ebf572894a55e767e9c57d1f1e914119 not found: ID does not exist"
Jul 30 14:02:20.546026 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.545994    2557 scope.go:117] "RemoveContainer" containerID="cb71fae6013ac559d583a517ee5cfb54a616f51b5b3ca72f55d39506c77481dc"
Jul 30 14:02:20.546207 ip-10-0-143-33 kubenswrapper[2557]: E0730 14:02:20.546188    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb71fae6013ac559d583a517ee5cfb54a616f51b5b3ca72f55d39506c77481dc\": container with ID starting with cb71fae6013ac559d583a517ee5cfb54a616f51b5b3ca72f55d39506c77481dc not found: ID does not exist" containerID="cb71fae6013ac559d583a517ee5cfb54a616f51b5b3ca72f55d39506c77481dc"
Jul 30 14:02:20.546270 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.546219    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb71fae6013ac559d583a517ee5cfb54a616f51b5b3ca72f55d39506c77481dc"} err="failed to get container status \"cb71fae6013ac559d583a517ee5cfb54a616f51b5b3ca72f55d39506c77481dc\": rpc error: code = NotFound desc = could not find container \"cb71fae6013ac559d583a517ee5cfb54a616f51b5b3ca72f55d39506c77481dc\": container with ID starting with cb71fae6013ac559d583a517ee5cfb54a616f51b5b3ca72f55d39506c77481dc not found: ID does not exist"
Jul 30 14:02:20.546270 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.546242    2557 scope.go:117] "RemoveContainer" containerID="3d73733abe46e63d42693e5748b4167173c31c7289888bb02c39deafcc4004d4"
Jul 30 14:02:20.551903 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.551882    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"]
Jul 30 14:02:20.555473 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.555445    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-cd11f9-openshift-default-544b7d8c5d-28j5h"]
Jul 30 14:02:20.555799 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.555786    2557 scope.go:117] "RemoveContainer" containerID="3d73733abe46e63d42693e5748b4167173c31c7289888bb02c39deafcc4004d4"
Jul 30 14:02:20.556098 ip-10-0-143-33 kubenswrapper[2557]: E0730 14:02:20.556078    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d73733abe46e63d42693e5748b4167173c31c7289888bb02c39deafcc4004d4\": container with ID starting with 3d73733abe46e63d42693e5748b4167173c31c7289888bb02c39deafcc4004d4 not found: ID does not exist" containerID="3d73733abe46e63d42693e5748b4167173c31c7289888bb02c39deafcc4004d4"
Jul 30 14:02:20.556166 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.556104    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d73733abe46e63d42693e5748b4167173c31c7289888bb02c39deafcc4004d4"} err="failed to get container status \"3d73733abe46e63d42693e5748b4167173c31c7289888bb02c39deafcc4004d4\": rpc error: code = NotFound desc = could not find container \"3d73733abe46e63d42693e5748b4167173c31c7289888bb02c39deafcc4004d4\": container with ID starting with 3d73733abe46e63d42693e5748b4167173c31c7289888bb02c39deafcc4004d4 not found: ID does not exist"
Jul 30 14:02:20.566555 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.566533    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"]
Jul 30 14:02:20.572968 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.572943    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5bef-5b9c4c65cc-t4z7n"]
Jul 30 14:02:20.606051 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.606021    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b346b520-732e-413f-beb2-27fd874cc12c-openshift-service-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:20.606051 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.606058    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b346b520-732e-413f-beb2-27fd874cc12c-maas-api-tls\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:20.606209 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:20.606074    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsfmd\" (UniqueName: \"kubernetes.io/projected/b346b520-732e-413f-beb2-27fd874cc12c-kube-api-access-wsfmd\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:21.183249 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:21.183212    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b346b520-732e-413f-beb2-27fd874cc12c" path="/var/lib/kubelet/pods/b346b520-732e-413f-beb2-27fd874cc12c/volumes"
Jul 30 14:02:21.183675 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:21.183660    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c08da357-1adf-4e23-a937-bf3c971dcb47" path="/var/lib/kubelet/pods/c08da357-1adf-4e23-a937-bf3c971dcb47/volumes"
Jul 30 14:02:23.338411 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:23.338370    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 14:02:27.167506 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.167464    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc"]
Jul 30 14:02:27.168064 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168044    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b346b520-732e-413f-beb2-27fd874cc12c" containerName="maas-api"
Jul 30 14:02:27.168147 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168068    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b346b520-732e-413f-beb2-27fd874cc12c" containerName="maas-api"
Jul 30 14:02:27.168147 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168090    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c08da357-1adf-4e23-a937-bf3c971dcb47" containerName="istio-proxy"
Jul 30 14:02:27.168147 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168099    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c08da357-1adf-4e23-a937-bf3c971dcb47" containerName="istio-proxy"
Jul 30 14:02:27.168147 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168111    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c08da357-1adf-4e23-a937-bf3c971dcb47" containerName="istio-proxy"
Jul 30 14:02:27.168147 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168120    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c08da357-1adf-4e23-a937-bf3c971dcb47" containerName="istio-proxy"
Jul 30 14:02:27.168147 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168129    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b19ecb58-ff4e-46d3-a0d7-a034ad2409c9" containerName="maas-api"
Jul 30 14:02:27.168147 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168138    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b19ecb58-ff4e-46d3-a0d7-a034ad2409c9" containerName="maas-api"
Jul 30 14:02:27.168490 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168155    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c68172-6fea-4c22-9514-5868f8828a7f" containerName="main"
Jul 30 14:02:27.168490 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168163    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c68172-6fea-4c22-9514-5868f8828a7f" containerName="main"
Jul 30 14:02:27.168490 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168174    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95b53ace-c9a9-4e3f-8731-109cd04b1b3a" containerName="maas-api"
Jul 30 14:02:27.168490 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168182    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="95b53ace-c9a9-4e3f-8731-109cd04b1b3a" containerName="maas-api"
Jul 30 14:02:27.168490 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168298    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="95b53ace-c9a9-4e3f-8731-109cd04b1b3a" containerName="maas-api"
Jul 30 14:02:27.168490 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168311    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b346b520-732e-413f-beb2-27fd874cc12c" containerName="maas-api"
Jul 30 14:02:27.168490 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168321    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="59c68172-6fea-4c22-9514-5868f8828a7f" containerName="main"
Jul 30 14:02:27.168490 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168331    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b19ecb58-ff4e-46d3-a0d7-a034ad2409c9" containerName="maas-api"
Jul 30 14:02:27.168490 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.168341    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c08da357-1adf-4e23-a937-bf3c971dcb47" containerName="istio-proxy"
Jul 30 14:02:27.171524 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.171504    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc"
Jul 30 14:02:27.174403 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.174384    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxdpz\""
Jul 30 14:02:27.186374 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.186353    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc"]
Jul 30 14:02:27.268943 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.268896    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5afcc821-09e4-4e91-a200-98b667ff17d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc\" (UID: \"5afcc821-09e4-4e91-a200-98b667ff17d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc"
Jul 30 14:02:27.370134 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.370095    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5afcc821-09e4-4e91-a200-98b667ff17d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc\" (UID: \"5afcc821-09e4-4e91-a200-98b667ff17d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc"
Jul 30 14:02:27.370747 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.370729    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5afcc821-09e4-4e91-a200-98b667ff17d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc\" (UID: \"5afcc821-09e4-4e91-a200-98b667ff17d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc"
Jul 30 14:02:27.483820 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.483737    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc"
Jul 30 14:02:27.630947 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:27.630889    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc"]
Jul 30 14:02:27.633324 ip-10-0-143-33 kubenswrapper[2557]: W0730 14:02:27.633293    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5afcc821_09e4_4e91_a200_98b667ff17d6.slice/crio-81c792744875cb714acfad830ad01b7c2e32ae33bd111c9702d3982babcd8206 WatchSource:0}: Error finding container 81c792744875cb714acfad830ad01b7c2e32ae33bd111c9702d3982babcd8206: Status 404 returned error can't find the container with id 81c792744875cb714acfad830ad01b7c2e32ae33bd111c9702d3982babcd8206
Jul 30 14:02:28.569383 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:28.569350    2557 generic.go:356] "Generic (PLEG): container finished" podID="5afcc821-09e4-4e91-a200-98b667ff17d6" containerID="cc4ea5156c8c806ce2f1a4a49743a0e3cd96c704b2588885a27c486c62561852" exitCode=0
Jul 30 14:02:28.569802 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:28.569437    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc" event={"ID":"5afcc821-09e4-4e91-a200-98b667ff17d6","Type":"ContainerDied","Data":"cc4ea5156c8c806ce2f1a4a49743a0e3cd96c704b2588885a27c486c62561852"}
Jul 30 14:02:28.569802 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:28.569486    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc" event={"ID":"5afcc821-09e4-4e91-a200-98b667ff17d6","Type":"ContainerStarted","Data":"81c792744875cb714acfad830ad01b7c2e32ae33bd111c9702d3982babcd8206"}
Jul 30 14:02:29.700179 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:29.700152    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc"
Jul 30 14:02:29.791928 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:29.791893    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5afcc821-09e4-4e91-a200-98b667ff17d6-maas-api-service-ca\") pod \"5afcc821-09e4-4e91-a200-98b667ff17d6\" (UID: \"5afcc821-09e4-4e91-a200-98b667ff17d6\") "
Jul 30 14:02:29.792276 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:29.792250    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5afcc821-09e4-4e91-a200-98b667ff17d6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5afcc821-09e4-4e91-a200-98b667ff17d6" (UID: "5afcc821-09e4-4e91-a200-98b667ff17d6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 14:02:29.893198 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:29.893117    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5afcc821-09e4-4e91-a200-98b667ff17d6-maas-api-service-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:30.579521 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:30.579492    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc"
Jul 30 14:02:30.579721 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:30.579533    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc" event={"ID":"5afcc821-09e4-4e91-a200-98b667ff17d6","Type":"ContainerDied","Data":"81c792744875cb714acfad830ad01b7c2e32ae33bd111c9702d3982babcd8206"}
Jul 30 14:02:30.579721 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:30.579580    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81c792744875cb714acfad830ad01b7c2e32ae33bd111c9702d3982babcd8206"
Jul 30 14:02:33.423735 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.423702    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"]
Jul 30 14:02:33.424215 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.423910    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm" podUID="71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485" containerName="maas-api" containerID="cri-o://77f1f91de5847054ef1cc870a78e4c0affcad5ad948a113b61ac9243af67a8f9" gracePeriod=30
Jul 30 14:02:33.597860 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.597810    2557 generic.go:356] "Generic (PLEG): container finished" podID="71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485" containerID="77f1f91de5847054ef1cc870a78e4c0affcad5ad948a113b61ac9243af67a8f9" exitCode=0
Jul 30 14:02:33.597994 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.597859    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm" event={"ID":"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485","Type":"ContainerDied","Data":"77f1f91de5847054ef1cc870a78e4c0affcad5ad948a113b61ac9243af67a8f9"}
Jul 30 14:02:33.625797 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.625766    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc"]
Jul 30 14:02:33.629852 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.629826    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f4dc0a-s7bnc"]
Jul 30 14:02:33.679988 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.679923    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 14:02:33.830724 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.830691    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-maas-api-tls\") pod \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\" (UID: \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\") "
Jul 30 14:02:33.830894 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.830758    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8dt7\" (UniqueName: \"kubernetes.io/projected/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-kube-api-access-h8dt7\") pod \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\" (UID: \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\") "
Jul 30 14:02:33.830894 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.830846    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-openshift-service-ca\") pod \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\" (UID: \"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485\") "
Jul 30 14:02:33.831215 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.831191    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485" (UID: "71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 14:02:33.832781 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.832754    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485" (UID: "71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 14:02:33.832885 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.832865    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-kube-api-access-h8dt7" (OuterVolumeSpecName: "kube-api-access-h8dt7") pod "71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485" (UID: "71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485"). InnerVolumeSpecName "kube-api-access-h8dt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:02:33.932432 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.932352    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-maas-api-tls\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:33.932432 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.932381    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8dt7\" (UniqueName: \"kubernetes.io/projected/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-kube-api-access-h8dt7\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:33.932432 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:33.932390    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485-openshift-service-ca\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:34.604440 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:34.604409    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"
Jul 30 14:02:34.604924 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:34.604446    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm" event={"ID":"71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485","Type":"ContainerDied","Data":"be5b2307e9b39b4eecee57358c772fde149a5a7fbc3cee8c436089a8f86187be"}
Jul 30 14:02:34.604924 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:34.604514    2557 scope.go:117] "RemoveContainer" containerID="77f1f91de5847054ef1cc870a78e4c0affcad5ad948a113b61ac9243af67a8f9"
Jul 30 14:02:34.628246 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:34.628217    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"]
Jul 30 14:02:34.631245 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:34.631225    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f4dc0a-74f846cdd7-nsqzm"]
Jul 30 14:02:35.182758 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:35.182728    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5afcc821-09e4-4e91-a200-98b667ff17d6" path="/var/lib/kubelet/pods/5afcc821-09e4-4e91-a200-98b667ff17d6/volumes"
Jul 30 14:02:35.183129 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:35.183117    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485" path="/var/lib/kubelet/pods/71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485/volumes"
Jul 30 14:02:36.841742 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:36.839795    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 14:02:40.159466 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:40.159429    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"]
Jul 30 14:02:40.159904 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:40.159686    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc" podUID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerName="istio-proxy" containerID="cri-o://1f8a9c551da3e21bc665a4018ef832cddbdaa8aae5106268c76c296aff10d047" gracePeriod=30
Jul 30 14:02:43.610333 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:43.610289    2557 patch_prober.go:28] interesting pod/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 14:02:43.610755 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:43.610350    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc" podUID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 14:02:45.427946 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.427883    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 14:02:45.433487 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.433465    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-workload-socket\") pod \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") "
Jul 30 14:02:45.433627 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.433497    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istiod-ca-cert\") pod \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") "
Jul 30 14:02:45.433627 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.433522    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-token\") pod \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") "
Jul 30 14:02:45.433627 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.433563    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-workload-certs\") pod \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") "
Jul 30 14:02:45.433627 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.433614    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-podinfo\") pod \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") "
Jul 30 14:02:45.433873 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.433668    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxjpw\" (UniqueName: \"kubernetes.io/projected/d1a35c54-1b67-4918-bdea-5b1a270b0a34-kube-api-access-wxjpw\") pod \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") "
Jul 30 14:02:45.433873 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.433705    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-credential-socket\") pod \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") "
Jul 30 14:02:45.433873 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.433742    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-data\") pod \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") "
Jul 30 14:02:45.433873 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.433783    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-envoy\") pod \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\" (UID: \"d1a35c54-1b67-4918-bdea-5b1a270b0a34\") "
Jul 30 14:02:45.433873 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.433785    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d1a35c54-1b67-4918-bdea-5b1a270b0a34" (UID: "d1a35c54-1b67-4918-bdea-5b1a270b0a34"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:02:45.434129 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.433893    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d1a35c54-1b67-4918-bdea-5b1a270b0a34" (UID: "d1a35c54-1b67-4918-bdea-5b1a270b0a34"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 14:02:45.434129 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.434027    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d1a35c54-1b67-4918-bdea-5b1a270b0a34" (UID: "d1a35c54-1b67-4918-bdea-5b1a270b0a34"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:02:45.434129 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.434044    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d1a35c54-1b67-4918-bdea-5b1a270b0a34" (UID: "d1a35c54-1b67-4918-bdea-5b1a270b0a34"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:02:45.434286 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.434134    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-workload-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:45.434286 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.434151    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-credential-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:45.434286 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.434165    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-workload-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:45.434286 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.434177    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istiod-ca-cert\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:45.434632 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.434611    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-data" (OuterVolumeSpecName: "istio-data") pod "d1a35c54-1b67-4918-bdea-5b1a270b0a34" (UID: "d1a35c54-1b67-4918-bdea-5b1a270b0a34"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:02:45.435873 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.435839    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-token" (OuterVolumeSpecName: "istio-token") pod "d1a35c54-1b67-4918-bdea-5b1a270b0a34" (UID: "d1a35c54-1b67-4918-bdea-5b1a270b0a34"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:02:45.435997 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.435919    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d1a35c54-1b67-4918-bdea-5b1a270b0a34" (UID: "d1a35c54-1b67-4918-bdea-5b1a270b0a34"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 14:02:45.436200 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.436178    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1a35c54-1b67-4918-bdea-5b1a270b0a34-kube-api-access-wxjpw" (OuterVolumeSpecName: "kube-api-access-wxjpw") pod "d1a35c54-1b67-4918-bdea-5b1a270b0a34" (UID: "d1a35c54-1b67-4918-bdea-5b1a270b0a34"). InnerVolumeSpecName "kube-api-access-wxjpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:02:45.436200 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.436183    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d1a35c54-1b67-4918-bdea-5b1a270b0a34" (UID: "d1a35c54-1b67-4918-bdea-5b1a270b0a34"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:02:45.534873 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.534838    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-token\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:45.534873 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.534865    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-podinfo\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:45.534873 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.534874    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxjpw\" (UniqueName: \"kubernetes.io/projected/d1a35c54-1b67-4918-bdea-5b1a270b0a34-kube-api-access-wxjpw\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:45.534873 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.534883    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-data\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:45.535146 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.534892    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1a35c54-1b67-4918-bdea-5b1a270b0a34-istio-envoy\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:02:45.659658 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.659619    2557 generic.go:356] "Generic (PLEG): container finished" podID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerID="1f8a9c551da3e21bc665a4018ef832cddbdaa8aae5106268c76c296aff10d047" exitCode=0
Jul 30 14:02:45.659829 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.659688    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"
Jul 30 14:02:45.659829 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.659715    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc" event={"ID":"d1a35c54-1b67-4918-bdea-5b1a270b0a34","Type":"ContainerDied","Data":"1f8a9c551da3e21bc665a4018ef832cddbdaa8aae5106268c76c296aff10d047"}
Jul 30 14:02:45.659829 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.659766    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc" event={"ID":"d1a35c54-1b67-4918-bdea-5b1a270b0a34","Type":"ContainerDied","Data":"8b440756700dd5de25a3259f0c4821474ae970ec34decea7b9f311c2d22572fa"}
Jul 30 14:02:45.659829 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.659789    2557 scope.go:117] "RemoveContainer" containerID="1f8a9c551da3e21bc665a4018ef832cddbdaa8aae5106268c76c296aff10d047"
Jul 30 14:02:45.668752 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.668734    2557 scope.go:117] "RemoveContainer" containerID="cba115ce1db2a6771bbc2a766e98a07fcd67fcea6ccf46214c92ebdb56115a34"
Jul 30 14:02:45.676448 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.676429    2557 scope.go:117] "RemoveContainer" containerID="1f8a9c551da3e21bc665a4018ef832cddbdaa8aae5106268c76c296aff10d047"
Jul 30 14:02:45.676748 ip-10-0-143-33 kubenswrapper[2557]: E0730 14:02:45.676727    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f8a9c551da3e21bc665a4018ef832cddbdaa8aae5106268c76c296aff10d047\": container with ID starting with 1f8a9c551da3e21bc665a4018ef832cddbdaa8aae5106268c76c296aff10d047 not found: ID does not exist" containerID="1f8a9c551da3e21bc665a4018ef832cddbdaa8aae5106268c76c296aff10d047"
Jul 30 14:02:45.676849 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.676762    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f8a9c551da3e21bc665a4018ef832cddbdaa8aae5106268c76c296aff10d047"} err="failed to get container status \"1f8a9c551da3e21bc665a4018ef832cddbdaa8aae5106268c76c296aff10d047\": rpc error: code = NotFound desc = could not find container \"1f8a9c551da3e21bc665a4018ef832cddbdaa8aae5106268c76c296aff10d047\": container with ID starting with 1f8a9c551da3e21bc665a4018ef832cddbdaa8aae5106268c76c296aff10d047 not found: ID does not exist"
Jul 30 14:02:45.676849 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.676785    2557 scope.go:117] "RemoveContainer" containerID="cba115ce1db2a6771bbc2a766e98a07fcd67fcea6ccf46214c92ebdb56115a34"
Jul 30 14:02:45.677035 ip-10-0-143-33 kubenswrapper[2557]: E0730 14:02:45.677017    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cba115ce1db2a6771bbc2a766e98a07fcd67fcea6ccf46214c92ebdb56115a34\": container with ID starting with cba115ce1db2a6771bbc2a766e98a07fcd67fcea6ccf46214c92ebdb56115a34 not found: ID does not exist" containerID="cba115ce1db2a6771bbc2a766e98a07fcd67fcea6ccf46214c92ebdb56115a34"
Jul 30 14:02:45.677076 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.677041    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cba115ce1db2a6771bbc2a766e98a07fcd67fcea6ccf46214c92ebdb56115a34"} err="failed to get container status \"cba115ce1db2a6771bbc2a766e98a07fcd67fcea6ccf46214c92ebdb56115a34\": rpc error: code = NotFound desc = could not find container \"cba115ce1db2a6771bbc2a766e98a07fcd67fcea6ccf46214c92ebdb56115a34\": container with ID starting with cba115ce1db2a6771bbc2a766e98a07fcd67fcea6ccf46214c92ebdb56115a34 not found: ID does not exist"
Jul 30 14:02:45.681897 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.681848    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"]
Jul 30 14:02:45.685413 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:45.685393    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-f4dc0a-openshift-default-65f576947b-h6kqc"]
Jul 30 14:02:47.190215 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:47.190180    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" path="/var/lib/kubelet/pods/d1a35c54-1b67-4918-bdea-5b1a270b0a34/volumes"
Jul 30 14:02:53.832377 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:53.832346    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcmbj"]
Jul 30 14:02:57.855606 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:57.855562    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"]
Jul 30 14:02:57.856092 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:57.855793    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z" podUID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerName="istio-proxy" containerID="cri-o://fc4f7060e19687ab161f0df182fff36d7bc7731466a07f6471826f5a0539ab12" gracePeriod=30
Jul 30 14:02:57.918957 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:57.918919    2557 patch_prober.go:28] interesting pod/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 14:02:57.919115 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:02:57.918987    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z" podUID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 14:03:03.104808 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.104782    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 14:03:03.167235 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.167204    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-token\") pod \"e6258642-5130-4e8b-ad53-2bdb682f8b29\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") "
Jul 30 14:03:03.167407 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.167243    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6258642-5130-4e8b-ad53-2bdb682f8b29-istiod-ca-cert\") pod \"e6258642-5130-4e8b-ad53-2bdb682f8b29\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") "
Jul 30 14:03:03.167407 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.167304    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-credential-socket\") pod \"e6258642-5130-4e8b-ad53-2bdb682f8b29\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") "
Jul 30 14:03:03.167407 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.167358    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-envoy\") pod \"e6258642-5130-4e8b-ad53-2bdb682f8b29\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") "
Jul 30 14:03:03.167407 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.167403    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-podinfo\") pod \"e6258642-5130-4e8b-ad53-2bdb682f8b29\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") "
Jul 30 14:03:03.167675 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.167426    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-data\") pod \"e6258642-5130-4e8b-ad53-2bdb682f8b29\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") "
Jul 30 14:03:03.167675 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.167457    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fvfw\" (UniqueName: \"kubernetes.io/projected/e6258642-5130-4e8b-ad53-2bdb682f8b29-kube-api-access-6fvfw\") pod \"e6258642-5130-4e8b-ad53-2bdb682f8b29\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") "
Jul 30 14:03:03.167675 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.167500    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-workload-certs\") pod \"e6258642-5130-4e8b-ad53-2bdb682f8b29\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") "
Jul 30 14:03:03.167675 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.167571    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-workload-socket\") pod \"e6258642-5130-4e8b-ad53-2bdb682f8b29\" (UID: \"e6258642-5130-4e8b-ad53-2bdb682f8b29\") "
Jul 30 14:03:03.167873 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.167715    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6258642-5130-4e8b-ad53-2bdb682f8b29-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e6258642-5130-4e8b-ad53-2bdb682f8b29" (UID: "e6258642-5130-4e8b-ad53-2bdb682f8b29"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 14:03:03.167936 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.167888    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6258642-5130-4e8b-ad53-2bdb682f8b29-istiod-ca-cert\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:03:03.168017 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.167935    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e6258642-5130-4e8b-ad53-2bdb682f8b29" (UID: "e6258642-5130-4e8b-ad53-2bdb682f8b29"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:03:03.168076 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.168001    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e6258642-5130-4e8b-ad53-2bdb682f8b29" (UID: "e6258642-5130-4e8b-ad53-2bdb682f8b29"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:03:03.168121 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.168088    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e6258642-5130-4e8b-ad53-2bdb682f8b29" (UID: "e6258642-5130-4e8b-ad53-2bdb682f8b29"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:03:03.168420 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.168393    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-data" (OuterVolumeSpecName: "istio-data") pod "e6258642-5130-4e8b-ad53-2bdb682f8b29" (UID: "e6258642-5130-4e8b-ad53-2bdb682f8b29"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:03:03.169583 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.169560    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e6258642-5130-4e8b-ad53-2bdb682f8b29" (UID: "e6258642-5130-4e8b-ad53-2bdb682f8b29"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 14:03:03.169681 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.169588    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-token" (OuterVolumeSpecName: "istio-token") pod "e6258642-5130-4e8b-ad53-2bdb682f8b29" (UID: "e6258642-5130-4e8b-ad53-2bdb682f8b29"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:03:03.169895 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.169880    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e6258642-5130-4e8b-ad53-2bdb682f8b29" (UID: "e6258642-5130-4e8b-ad53-2bdb682f8b29"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:03:03.169989 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.169968    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6258642-5130-4e8b-ad53-2bdb682f8b29-kube-api-access-6fvfw" (OuterVolumeSpecName: "kube-api-access-6fvfw") pod "e6258642-5130-4e8b-ad53-2bdb682f8b29" (UID: "e6258642-5130-4e8b-ad53-2bdb682f8b29"). InnerVolumeSpecName "kube-api-access-6fvfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:03:03.268270 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.268226    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-credential-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:03:03.268270 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.268267    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-envoy\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:03:03.268270 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.268277    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-podinfo\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:03:03.268497 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.268284    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-data\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:03:03.268497 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.268293    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fvfw\" (UniqueName: \"kubernetes.io/projected/e6258642-5130-4e8b-ad53-2bdb682f8b29-kube-api-access-6fvfw\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:03:03.268497 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.268301    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-workload-certs\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:03:03.268497 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.268309    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6258642-5130-4e8b-ad53-2bdb682f8b29-workload-socket\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:03:03.268497 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.268316    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6258642-5130-4e8b-ad53-2bdb682f8b29-istio-token\") on node \"ip-10-0-143-33.ec2.internal\" DevicePath \"\""
Jul 30 14:03:03.748580 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.748542    2557 generic.go:356] "Generic (PLEG): container finished" podID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerID="fc4f7060e19687ab161f0df182fff36d7bc7731466a07f6471826f5a0539ab12" exitCode=0
Jul 30 14:03:03.748807 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.748613    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"
Jul 30 14:03:03.748807 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.748628    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z" event={"ID":"e6258642-5130-4e8b-ad53-2bdb682f8b29","Type":"ContainerDied","Data":"fc4f7060e19687ab161f0df182fff36d7bc7731466a07f6471826f5a0539ab12"}
Jul 30 14:03:03.748807 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.748674    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z" event={"ID":"e6258642-5130-4e8b-ad53-2bdb682f8b29","Type":"ContainerDied","Data":"d8862b1de37874b59ee8bb18a39aa8a7a623bac5e5730e1ad9a4f3811510908e"}
Jul 30 14:03:03.748807 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.748698    2557 scope.go:117] "RemoveContainer" containerID="fc4f7060e19687ab161f0df182fff36d7bc7731466a07f6471826f5a0539ab12"
Jul 30 14:03:03.756955 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.756935    2557 scope.go:117] "RemoveContainer" containerID="0b6418807394f4dcf46f9175a3e944525aca761d1cb9ba972c39e7867fc9458a"
Jul 30 14:03:03.764566 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.764548    2557 scope.go:117] "RemoveContainer" containerID="fc4f7060e19687ab161f0df182fff36d7bc7731466a07f6471826f5a0539ab12"
Jul 30 14:03:03.764852 ip-10-0-143-33 kubenswrapper[2557]: E0730 14:03:03.764834    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc4f7060e19687ab161f0df182fff36d7bc7731466a07f6471826f5a0539ab12\": container with ID starting with fc4f7060e19687ab161f0df182fff36d7bc7731466a07f6471826f5a0539ab12 not found: ID does not exist" containerID="fc4f7060e19687ab161f0df182fff36d7bc7731466a07f6471826f5a0539ab12"
Jul 30 14:03:03.764915 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.764861    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc4f7060e19687ab161f0df182fff36d7bc7731466a07f6471826f5a0539ab12"} err="failed to get container status \"fc4f7060e19687ab161f0df182fff36d7bc7731466a07f6471826f5a0539ab12\": rpc error: code = NotFound desc = could not find container \"fc4f7060e19687ab161f0df182fff36d7bc7731466a07f6471826f5a0539ab12\": container with ID starting with fc4f7060e19687ab161f0df182fff36d7bc7731466a07f6471826f5a0539ab12 not found: ID does not exist"
Jul 30 14:03:03.764915 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.764876    2557 scope.go:117] "RemoveContainer" containerID="0b6418807394f4dcf46f9175a3e944525aca761d1cb9ba972c39e7867fc9458a"
Jul 30 14:03:03.765117 ip-10-0-143-33 kubenswrapper[2557]: E0730 14:03:03.765098    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b6418807394f4dcf46f9175a3e944525aca761d1cb9ba972c39e7867fc9458a\": container with ID starting with 0b6418807394f4dcf46f9175a3e944525aca761d1cb9ba972c39e7867fc9458a not found: ID does not exist" containerID="0b6418807394f4dcf46f9175a3e944525aca761d1cb9ba972c39e7867fc9458a"
Jul 30 14:03:03.765171 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.765124    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b6418807394f4dcf46f9175a3e944525aca761d1cb9ba972c39e7867fc9458a"} err="failed to get container status \"0b6418807394f4dcf46f9175a3e944525aca761d1cb9ba972c39e7867fc9458a\": rpc error: code = NotFound desc = could not find container \"0b6418807394f4dcf46f9175a3e944525aca761d1cb9ba972c39e7867fc9458a\": container with ID starting with 0b6418807394f4dcf46f9175a3e944525aca761d1cb9ba972c39e7867fc9458a not found: ID does not exist"
Jul 30 14:03:03.767980 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.767960    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"]
Jul 30 14:03:03.774659 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:03.774639    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-e945a1-openshift-default-5dcfc45bdd-hsc8z"]
Jul 30 14:03:05.182572 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:05.182538    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6258642-5130-4e8b-ad53-2bdb682f8b29" path="/var/lib/kubelet/pods/e6258642-5130-4e8b-ad53-2bdb682f8b29/volumes"
Jul 30 14:03:05.540773 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:05.540698    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-b8b49f4d5-rzf74_6ddbc56d-98ea-4947-ad78-478d54248d40/manager/0.log"
Jul 30 14:03:06.150414 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:06.150383    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6_d9cda7fe-e008-4ffe-9f50-0a87e31e53ce/copy-manifests/0.log"
Jul 30 14:03:06.168455 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:06.168425    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6_d9cda7fe-e008-4ffe-9f50-0a87e31e53ce/manager/0.log"
Jul 30 14:03:06.661985 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:06.661961    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29756970-czp7p_be7661ce-26e5-43a3-a727-9e31e654190a/cleanup/0.log"
Jul 30 14:03:06.776698 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:06.776662    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29756985-ljc9r_82579774-d905-4280-8886-bb44ea4570d4/cleanup/0.log"
Jul 30 14:03:08.836899 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:08.836867    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-5t68l_07468d65-d747-4283-8535-29d78f872f68/kuadrant-console-plugin/0.log"
Jul 30 14:03:09.208827 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:09.208798    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-bcmbj_98e36e50-caba-4e26-a2b3-082265e732ac/limitador/0.log"
Jul 30 14:03:10.099494 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:10.099463    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54d6c599cc-b8lfk_4a743ebf-e046-43c5-8e66-ef50e90ce5b5/kube-auth-proxy/0.log"
Jul 30 14:03:17.815101 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815066    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nd8l9/must-gather-6wcfj"]
Jul 30 14:03:17.815501 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815398    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerName="istio-proxy"
Jul 30 14:03:17.815501 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815409    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerName="istio-proxy"
Jul 30 14:03:17.815501 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815420    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5afcc821-09e4-4e91-a200-98b667ff17d6" containerName="revoke-keys"
Jul 30 14:03:17.815501 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815425    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5afcc821-09e4-4e91-a200-98b667ff17d6" containerName="revoke-keys"
Jul 30 14:03:17.815501 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815434    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerName="istio-proxy"
Jul 30 14:03:17.815501 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815440    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerName="istio-proxy"
Jul 30 14:03:17.815501 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815447    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerName="istio-proxy"
Jul 30 14:03:17.815501 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815452    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerName="istio-proxy"
Jul 30 14:03:17.815501 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815468    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485" containerName="maas-api"
Jul 30 14:03:17.815501 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815474    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485" containerName="maas-api"
Jul 30 14:03:17.815829 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815525    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c08da357-1adf-4e23-a937-bf3c971dcb47" containerName="istio-proxy"
Jul 30 14:03:17.815829 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815533    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e0f5c4-b2c6-4079-9cd9-5c2b52ea4485" containerName="maas-api"
Jul 30 14:03:17.815829 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815539    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerName="istio-proxy"
Jul 30 14:03:17.815829 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815545    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1a35c54-1b67-4918-bdea-5b1a270b0a34" containerName="istio-proxy"
Jul 30 14:03:17.815829 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815554    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerName="istio-proxy"
Jul 30 14:03:17.815829 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815559    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5afcc821-09e4-4e91-a200-98b667ff17d6" containerName="revoke-keys"
Jul 30 14:03:17.815829 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815565    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerName="istio-proxy"
Jul 30 14:03:17.815829 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815630    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerName="istio-proxy"
Jul 30 14:03:17.815829 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.815636    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6258642-5130-4e8b-ad53-2bdb682f8b29" containerName="istio-proxy"
Jul 30 14:03:17.819705 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.819688    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nd8l9/must-gather-6wcfj"
Jul 30 14:03:17.822445 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.822425    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nd8l9\"/\"kube-root-ca.crt\""
Jul 30 14:03:17.823750 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.823734    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nd8l9\"/\"default-dockercfg-fnqxp\""
Jul 30 14:03:17.823852 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.823752    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nd8l9\"/\"openshift-service-ca.crt\""
Jul 30 14:03:17.829050 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.829025    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nd8l9/must-gather-6wcfj"]
Jul 30 14:03:17.899471 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.899436    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwjg9\" (UniqueName: \"kubernetes.io/projected/3c9a5c35-75f9-48c6-b4c8-013ab0e52d83-kube-api-access-hwjg9\") pod \"must-gather-6wcfj\" (UID: \"3c9a5c35-75f9-48c6-b4c8-013ab0e52d83\") " pod="openshift-must-gather-nd8l9/must-gather-6wcfj"
Jul 30 14:03:17.899655 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:17.899559    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3c9a5c35-75f9-48c6-b4c8-013ab0e52d83-must-gather-output\") pod \"must-gather-6wcfj\" (UID: \"3c9a5c35-75f9-48c6-b4c8-013ab0e52d83\") " pod="openshift-must-gather-nd8l9/must-gather-6wcfj"
Jul 30 14:03:18.000749 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:18.000712    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwjg9\" (UniqueName: \"kubernetes.io/projected/3c9a5c35-75f9-48c6-b4c8-013ab0e52d83-kube-api-access-hwjg9\") pod \"must-gather-6wcfj\" (UID: \"3c9a5c35-75f9-48c6-b4c8-013ab0e52d83\") " pod="openshift-must-gather-nd8l9/must-gather-6wcfj"
Jul 30 14:03:18.000928 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:18.000799    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3c9a5c35-75f9-48c6-b4c8-013ab0e52d83-must-gather-output\") pod \"must-gather-6wcfj\" (UID: \"3c9a5c35-75f9-48c6-b4c8-013ab0e52d83\") " pod="openshift-must-gather-nd8l9/must-gather-6wcfj"
Jul 30 14:03:18.001101 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:18.001083    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3c9a5c35-75f9-48c6-b4c8-013ab0e52d83-must-gather-output\") pod \"must-gather-6wcfj\" (UID: \"3c9a5c35-75f9-48c6-b4c8-013ab0e52d83\") " pod="openshift-must-gather-nd8l9/must-gather-6wcfj"
Jul 30 14:03:18.017113 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:18.017089    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwjg9\" (UniqueName: \"kubernetes.io/projected/3c9a5c35-75f9-48c6-b4c8-013ab0e52d83-kube-api-access-hwjg9\") pod \"must-gather-6wcfj\" (UID: \"3c9a5c35-75f9-48c6-b4c8-013ab0e52d83\") " pod="openshift-must-gather-nd8l9/must-gather-6wcfj"
Jul 30 14:03:18.129765 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:18.129688    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nd8l9/must-gather-6wcfj"
Jul 30 14:03:18.263710 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:18.263678    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nd8l9/must-gather-6wcfj"]
Jul 30 14:03:18.267213 ip-10-0-143-33 kubenswrapper[2557]: W0730 14:03:18.267180    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c9a5c35_75f9_48c6_b4c8_013ab0e52d83.slice/crio-2899efa3828524da54e862f2da4b39f6adcea9f245eb52af9771af58a30d1576 WatchSource:0}: Error finding container 2899efa3828524da54e862f2da4b39f6adcea9f245eb52af9771af58a30d1576: Status 404 returned error can't find the container with id 2899efa3828524da54e862f2da4b39f6adcea9f245eb52af9771af58a30d1576
Jul 30 14:03:18.817339 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:18.817302    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nd8l9/must-gather-6wcfj" event={"ID":"3c9a5c35-75f9-48c6-b4c8-013ab0e52d83","Type":"ContainerStarted","Data":"2899efa3828524da54e862f2da4b39f6adcea9f245eb52af9771af58a30d1576"}
Jul 30 14:03:19.823825 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:19.823784    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nd8l9/must-gather-6wcfj" event={"ID":"3c9a5c35-75f9-48c6-b4c8-013ab0e52d83","Type":"ContainerStarted","Data":"724e0a1a2144f7c409e6d1c23bae8eb77d0e0c74d533c82f2ec815189c61964e"}
Jul 30 14:03:19.823825 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:19.823829    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nd8l9/must-gather-6wcfj" event={"ID":"3c9a5c35-75f9-48c6-b4c8-013ab0e52d83","Type":"ContainerStarted","Data":"c4e4f42dc1d21d09250ffb9c693daf6ff84bdd8b1314f87bb18d6755bd727b08"}
Jul 30 14:03:19.841318 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:19.841211    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nd8l9/must-gather-6wcfj" podStartSLOduration=1.523926557 podStartE2EDuration="2.841190811s" totalImagesPullingTime="1.317264254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:03:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:03:19.839752321 +0000 UTC m=+3823.112216737" watchObservedRunningTime="2026-07-30 14:03:19.841190811 +0000 UTC m=+3823.113655211"
Jul 30 14:03:21.113551 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:21.113514    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qbp52_26774edd-7404-4111-a22e-da6b38958c85/global-pull-secret-syncer/0.log"
Jul 30 14:03:21.249536 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:21.249498    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hn766_2a0f84c2-b0b6-4df9-ae88-8b98b4788961/konnectivity-agent/0.log"
Jul 30 14:03:21.362184 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:21.362141    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-33.ec2.internal_bd398727194bd86b778ce466d77f4b02/haproxy/0.log"
Jul 30 14:03:27.319889 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:27.319858    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-5t68l_07468d65-d747-4283-8535-29d78f872f68/kuadrant-console-plugin/0.log"
Jul 30 14:03:27.536271 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:27.536245    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-bcmbj_98e36e50-caba-4e26-a2b3-082265e732ac/limitador/0.log"
Jul 30 14:03:29.559474 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:29.559388    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6c9cb979f-pdhpb_21f1dec4-01ad-4e73-a3c5-a5244bbcfddb/metrics-server/0.log"
Jul 30 14:03:29.582498 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:29.582450    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-xd6pz_d57fc4c7-6da1-499b-9bd5-84450e3da020/monitoring-plugin/0.log"
Jul 30 14:03:29.611951 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:29.611913    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7k62v_c9c400f7-8b1b-4394-8226-fc7e317de93a/node-exporter/0.log"
Jul 30 14:03:29.634222 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:29.634194    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7k62v_c9c400f7-8b1b-4394-8226-fc7e317de93a/kube-rbac-proxy/0.log"
Jul 30 14:03:29.655483 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:29.655457    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7k62v_c9c400f7-8b1b-4394-8226-fc7e317de93a/init-textfile/0.log"
Jul 30 14:03:29.805255 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:29.805228    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-gg6kv_c136d17d-f123-49f0-aef4-990cfdea139c/kube-rbac-proxy-main/0.log"
Jul 30 14:03:29.823663 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:29.823570    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-gg6kv_c136d17d-f123-49f0-aef4-990cfdea139c/kube-rbac-proxy-self/0.log"
Jul 30 14:03:29.846793 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:29.846763    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-gg6kv_c136d17d-f123-49f0-aef4-990cfdea139c/openshift-state-metrics/0.log"
Jul 30 14:03:29.878522 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:29.878494    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_510e5683-2a6c-4b08-84d9-2680a2b3f824/prometheus/0.log"
Jul 30 14:03:29.896216 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:29.896184    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_510e5683-2a6c-4b08-84d9-2680a2b3f824/config-reloader/0.log"
Jul 30 14:03:29.919155 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:29.919126    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_510e5683-2a6c-4b08-84d9-2680a2b3f824/thanos-sidecar/0.log"
Jul 30 14:03:29.940218 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:29.940187    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_510e5683-2a6c-4b08-84d9-2680a2b3f824/kube-rbac-proxy-web/0.log"
Jul 30 14:03:29.960912 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:29.960884    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_510e5683-2a6c-4b08-84d9-2680a2b3f824/kube-rbac-proxy/0.log"
Jul 30 14:03:29.983424 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:29.983396    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_510e5683-2a6c-4b08-84d9-2680a2b3f824/kube-rbac-proxy-thanos/0.log"
Jul 30 14:03:30.002773 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:30.002743    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_510e5683-2a6c-4b08-84d9-2680a2b3f824/init-config-reloader/0.log"
Jul 30 14:03:30.177559 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:30.177527    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c6c58dd48-55hqk_dddb41df-4d00-4008-a164-6b21d1656174/thanos-query/0.log"
Jul 30 14:03:30.196485 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:30.196462    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c6c58dd48-55hqk_dddb41df-4d00-4008-a164-6b21d1656174/kube-rbac-proxy-web/0.log"
Jul 30 14:03:30.215266 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:30.215236    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c6c58dd48-55hqk_dddb41df-4d00-4008-a164-6b21d1656174/kube-rbac-proxy/0.log"
Jul 30 14:03:30.234130 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:30.234103    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c6c58dd48-55hqk_dddb41df-4d00-4008-a164-6b21d1656174/prom-label-proxy/0.log"
Jul 30 14:03:30.256071 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:30.256044    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c6c58dd48-55hqk_dddb41df-4d00-4008-a164-6b21d1656174/kube-rbac-proxy-rules/0.log"
Jul 30 14:03:30.279287 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:30.279259    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c6c58dd48-55hqk_dddb41df-4d00-4008-a164-6b21d1656174/kube-rbac-proxy-metrics/0.log"
Jul 30 14:03:31.066030 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.065971    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"]
Jul 30 14:03:31.074525 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.074494    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.081066 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.081034    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"]
Jul 30 14:03:31.145111 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.145076    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/31f76929-4f69-436d-aa8f-ba8df07bc9e2-proc\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.145430 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.145406    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gk62\" (UniqueName: \"kubernetes.io/projected/31f76929-4f69-436d-aa8f-ba8df07bc9e2-kube-api-access-2gk62\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.145578 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.145564    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/31f76929-4f69-436d-aa8f-ba8df07bc9e2-lib-modules\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.145809 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.145786    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31f76929-4f69-436d-aa8f-ba8df07bc9e2-sys\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.145908 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.145819    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/31f76929-4f69-436d-aa8f-ba8df07bc9e2-podres\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.246652 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.246585    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/31f76929-4f69-436d-aa8f-ba8df07bc9e2-proc\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.246846 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.246679    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gk62\" (UniqueName: \"kubernetes.io/projected/31f76929-4f69-436d-aa8f-ba8df07bc9e2-kube-api-access-2gk62\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.246846 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.246708    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/31f76929-4f69-436d-aa8f-ba8df07bc9e2-proc\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.246846 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.246717    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/31f76929-4f69-436d-aa8f-ba8df07bc9e2-lib-modules\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.246846 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.246821    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31f76929-4f69-436d-aa8f-ba8df07bc9e2-sys\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.246846 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.246842    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/31f76929-4f69-436d-aa8f-ba8df07bc9e2-lib-modules\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.247111 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.246851    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/31f76929-4f69-436d-aa8f-ba8df07bc9e2-podres\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.247111 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.246896    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31f76929-4f69-436d-aa8f-ba8df07bc9e2-sys\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.247111 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.246994    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/31f76929-4f69-436d-aa8f-ba8df07bc9e2-podres\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.255100 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.255071    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gk62\" (UniqueName: \"kubernetes.io/projected/31f76929-4f69-436d-aa8f-ba8df07bc9e2-kube-api-access-2gk62\") pod \"perf-node-gather-daemonset-9npvk\" (UID: \"31f76929-4f69-436d-aa8f-ba8df07bc9e2\") " pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.389101 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.388985    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.579022 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.578893    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"]
Jul 30 14:03:31.582960 ip-10-0-143-33 kubenswrapper[2557]: W0730 14:03:31.582909    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod31f76929_4f69_436d_aa8f_ba8df07bc9e2.slice/crio-ecd81fea7b83bad0b09a4062910f1f72aa9325de8b38338e34ad194d7dfb3224 WatchSource:0}: Error finding container ecd81fea7b83bad0b09a4062910f1f72aa9325de8b38338e34ad194d7dfb3224: Status 404 returned error can't find the container with id ecd81fea7b83bad0b09a4062910f1f72aa9325de8b38338e34ad194d7dfb3224
Jul 30 14:03:31.910643 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.910582    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk" event={"ID":"31f76929-4f69-436d-aa8f-ba8df07bc9e2","Type":"ContainerStarted","Data":"e8e2182395394141d818c039c5ee610d2ca25209a16d493a2f9d2ac4a366bdc7"}
Jul 30 14:03:31.910850 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.910675    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk" event={"ID":"31f76929-4f69-436d-aa8f-ba8df07bc9e2","Type":"ContainerStarted","Data":"ecd81fea7b83bad0b09a4062910f1f72aa9325de8b38338e34ad194d7dfb3224"}
Jul 30 14:03:31.910850 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.910785    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:31.929097 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:31.929027    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk" podStartSLOduration=0.929005661 podStartE2EDuration="929.005661ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:03:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:03:31.926267065 +0000 UTC m=+3835.198731463" watchObservedRunningTime="2026-07-30 14:03:31.929005661 +0000 UTC m=+3835.201470049"
Jul 30 14:03:33.037321 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:33.037294    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-sbk52_f9840c97-feb0-49d7-85b0-62a6de93d0b3/volume-data-source-validator/0.log"
Jul 30 14:03:34.070473 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:34.070415    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7tl52_00eda289-36d7-4d10-ab5e-849c4f33b5d7/dns/0.log"
Jul 30 14:03:34.089409 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:34.089386    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7tl52_00eda289-36d7-4d10-ab5e-849c4f33b5d7/kube-rbac-proxy/0.log"
Jul 30 14:03:34.170303 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:34.170275    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fng7k_bad62a71-0b34-453b-97c5-bb6ef94cabda/dns-node-resolver/0.log"
Jul 30 14:03:34.724650 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:34.724619    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-db6b6c6d9-72cq9_a8c98414-472d-47b8-9b14-275a45e8f39f/registry/0.log"
Jul 30 14:03:34.749291 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:34.749260    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5h277_e9b91626-80d1-487b-bdf6-75b892b2dd6b/node-ca/0.log"
Jul 30 14:03:35.955544 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:35.955509    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54d6c599cc-b8lfk_4a743ebf-e046-43c5-8e66-ef50e90ce5b5/kube-auth-proxy/0.log"
Jul 30 14:03:36.915749 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:36.915721    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dmnvl_080d61dc-1429-482d-bd36-a448dcd0ee30/serve-healthcheck-canary/0.log"
Jul 30 14:03:37.417979 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:37.417954    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-pr87k_5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0/insights-operator/0.log"
Jul 30 14:03:37.418627 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:37.418584    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-pr87k_5fc557b6-e9d4-4570-aec9-7e2e5f8eebe0/insights-operator/1.log"
Jul 30 14:03:37.550116 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:37.550084    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t47c8_47958414-0ee8-4833-9018-e8785b5ae94f/kube-rbac-proxy/0.log"
Jul 30 14:03:37.569429 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:37.569401    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t47c8_47958414-0ee8-4833-9018-e8785b5ae94f/exporter/0.log"
Jul 30 14:03:37.590342 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:37.590311    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t47c8_47958414-0ee8-4833-9018-e8785b5ae94f/extractor/0.log"
Jul 30 14:03:37.930639 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:37.930589    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nd8l9/perf-node-gather-daemonset-9npvk"
Jul 30 14:03:40.571320 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:40.571287    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-b8b49f4d5-rzf74_6ddbc56d-98ea-4947-ad78-478d54248d40/manager/0.log"
Jul 30 14:03:40.958085 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:40.957911    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6_d9cda7fe-e008-4ffe-9f50-0a87e31e53ce/manager/0.log"
Jul 30 14:03:40.975611 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:40.975520    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd55cb6f4-zvcw6_d9cda7fe-e008-4ffe-9f50-0a87e31e53ce/copy-manifests/0.log"
Jul 30 14:03:41.706425 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:41.706394    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54d6c599cc-b8lfk_4a743ebf-e046-43c5-8e66-ef50e90ce5b5/kube-auth-proxy/0.log"
Jul 30 14:03:42.254509 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:42.254474    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d8f5f8447-zhn9c_964335af-e13d-4f77-9a05-0e58dc53c637/manager/0.log"
Jul 30 14:03:43.420028 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:43.419979    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54d6c599cc-b8lfk_4a743ebf-e046-43c5-8e66-ef50e90ce5b5/kube-auth-proxy/0.log"
Jul 30 14:03:47.027079 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:47.027050    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-zx79q_226736e9-00a0-4887-b101-fa79c578eee9/migrator/0.log"
Jul 30 14:03:47.051415 ip-10-0-143-33 kubenswrapper[2557]: I0730 14:03:47.051388    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-zx79q_226736e9-00a0-4887-b101-fa79c578eee9/graceful-termination/0.log"