Sep 14 07:47:01.590875 ip-10-0-133-123 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 14 07:47:01.590889 ip-10-0-133-123 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 14 07:47:01.590915 ip-10-0-133-123 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 14 07:47:01.591201 ip-10-0-133-123 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 14 07:47:11.824277 ip-10-0-133-123 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 14 07:47:11.824293 ip-10-0-133-123 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 6aece2ec9b0d495593314562e0cc9371 --
Sep 14 07:49:05.168467 ip-10-0-133-123 systemd[1]: Starting Kubernetes Kubelet...
Sep 14 07:49:05.616308 ip-10-0-133-123 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:49:05.616308 ip-10-0-133-123 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 14 07:49:05.616308 ip-10-0-133-123 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:49:05.616762 ip-10-0-133-123 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 14 07:49:05.616762 ip-10-0-133-123 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:49:05.617285 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.617221    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 14 07:49:05.621377 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621362    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:05.621413 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621379    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:05.621413 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621383    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:05.621413 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621387    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:05.621413 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621390    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:05.621413 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621394    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:05.621413 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621397    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:05.621413 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621400    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:05.621413 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621403    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:05.621413 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621405    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:05.621413 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621408    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:05.621413 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621410    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:05.621413 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621413    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:05.621413 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621416    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:05.621413 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621419    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621422    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621425    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621427    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621430    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621432    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621436    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621439    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621441    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621444    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621446    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621449    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621451    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621454    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621456    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621459    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621461    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621464    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621466    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621469    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:05.621747 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621472    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621474    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621477    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621479    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621482    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621484    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621486    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621489    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621491    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621494    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621496    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621499    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621501    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621504    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621507    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621510    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621512    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621515    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621517    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621520    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:05.622304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621523    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621525    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621528    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621530    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621533    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621535    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621538    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621540    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621543    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621545    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621548    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621552    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621555    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621558    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621561    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621563    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621566    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621569    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621572    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:05.622791 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621574    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621577    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621580    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621584    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621588    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621591    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621593    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621597    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621600    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621603    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621605    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621608    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621611    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621615    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621618    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621621    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621624    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621626    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621629    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:05.623251 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621631    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621634    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621638    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621640    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621643    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621645    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.621648    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622321    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622328    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622331    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622334    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622338    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622341    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622344    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622346    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622349    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622352    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622354    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622357    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622360    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:05.623752 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622363    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622366    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622369    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622371    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622374    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622377    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622380    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622383    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622385    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622388    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622391    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622393    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622395    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622398    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622401    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622403    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622406    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622409    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622411    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:05.624252 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622414    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622416    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622419    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622422    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622424    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622427    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622430    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622433    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622435    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622438    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622441    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622443    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622445    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622449    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622453    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622455    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622458    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622461    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622464    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:05.624723 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622468    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622472    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622475    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622478    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622481    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622484    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622486    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622507    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622510    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622514    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622517    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622520    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622523    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622525    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622528    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622531    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622533    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622536    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622539    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622541    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:05.625186 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622544    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622547    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622549    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622552    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622555    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622559    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622564    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622567    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622570    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622573    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622575    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622578    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622580    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622583    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622586    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622588    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622592    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622596    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622599    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622602    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:05.625688 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622605    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622607    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622610    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622613    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622616    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622618    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622621    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622623    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622970    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622975    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622978    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622981    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622984    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622987    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622989    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622992    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622994    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.622997    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623000    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623003    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623006    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:05.626179 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623009    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623012    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623014    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623017    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623020    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623022    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623025    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623027    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623030    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623033    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623035    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623038    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623040    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623043    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623045    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623048    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623050    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623054    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623057    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:05.626700 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623060    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623063    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623066    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623069    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623071    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623075    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623078    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623080    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623083    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623085    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623089    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623092    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623094    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623097    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623099    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623102    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623105    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623108    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623111    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623113    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:05.627164 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623116    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623118    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623121    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623123    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623126    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623128    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623131    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623133    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623135    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623138    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623142    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623145    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623149    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623152    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623155    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623158    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623161    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623163    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623166    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:05.627682 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623169    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623172    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623175    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623179    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623182    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623185    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623188    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623190    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623193    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623196    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623198    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623201    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623203    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623219    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623222    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623225    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623227    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623230    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623232    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:05.628143 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623235    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623237    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623246    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623250    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623252    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623256    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623258    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623262    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623264    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623336    2576 flags.go:64] FLAG: --address="0.0.0.0"
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623381    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623389    2576 flags.go:64] FLAG: --anonymous-auth="true"
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623394    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623403    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623409    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623413    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623417    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623420    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623423    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623426    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623429    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623432    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 14 07:49:05.628642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623435    2576 flags.go:64] FLAG: --cgroup-root=""
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623438    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623441    2576 flags.go:64] FLAG: --client-ca-file=""
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623444    2576 flags.go:64] FLAG: --cloud-provider="external"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623446    2576 flags.go:64] FLAG: --cluster-dns="[]"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623450    2576 flags.go:64] FLAG: --cluster-domain=""
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623453    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623456    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623459    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623463    2576 flags.go:64] FLAG: --container-log-max-files="5"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623466    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623469    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623472    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623475    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623478    2576 flags.go:64] FLAG: --contention-profiling="false"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623481    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623484    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623487    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623490    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623496    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623499    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623502    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623505    2576 flags.go:64] FLAG: --enable-load-reader="false"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623508    2576 flags.go:64] FLAG: --enable-server="true"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623513    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 14 07:49:05.629203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623517    2576 flags.go:64] FLAG: --event-burst="100"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623520    2576 flags.go:64] FLAG: --event-qps="50"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623523    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623527    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623529    2576 flags.go:64] FLAG: --eviction-hard=""
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623533    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623536    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623539    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623542    2576 flags.go:64] FLAG: --eviction-soft=""
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623545    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623547    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623550    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623553    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623556    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623559    2576 flags.go:64] FLAG: --fail-swap-on="true"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623562    2576 flags.go:64] FLAG: --feature-gates=""
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623566    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623568    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623571    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623574    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623577    2576 flags.go:64] FLAG: --healthz-port="10248"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623580    2576 flags.go:64] FLAG: --help="false"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623583    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-133-123.ec2.internal"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623586    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 14 07:49:05.629835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623589    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623592    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623596    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623600    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623602    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623605    2576 flags.go:64] FLAG: --image-service-endpoint=""
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623608    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623613    2576 flags.go:64] FLAG: --kube-api-burst="100"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623616    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623619    2576 flags.go:64] FLAG: --kube-api-qps="50"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623622    2576 flags.go:64] FLAG: --kube-reserved=""
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623625    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623628    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623631    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623633    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623636    2576 flags.go:64] FLAG: --lock-file=""
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623639    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623642    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623645    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623650    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623653    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623656    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623659    2576 flags.go:64] FLAG: --logging-format="text"
Sep 14 07:49:05.630450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623662    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623665    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623668    2576 flags.go:64] FLAG: --manifest-url=""
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623671    2576 flags.go:64] FLAG: --manifest-url-header=""
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623675    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623678    2576 flags.go:64] FLAG: --max-open-files="1000000"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623682    2576 flags.go:64] FLAG: --max-pods="110"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623684    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623687    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623690    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623693    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623697    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623700    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623703    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623709    2576 flags.go:64] FLAG: --node-status-max-images="50"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623713    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623717    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623720    2576 flags.go:64] FLAG: --pod-cidr=""
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623723    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623728    2576 flags.go:64] FLAG: --pod-manifest-path=""
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623731    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623734    2576 flags.go:64] FLAG: --pods-per-core="0"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623737    2576 flags.go:64] FLAG: --port="10250"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623740    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 14 07:49:05.631000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623743    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b027ba191ef7d892"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623746    2576 flags.go:64] FLAG: --qos-reserved=""
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623749    2576 flags.go:64] FLAG: --read-only-port="10255"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623752    2576 flags.go:64] FLAG: --register-node="true"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623755    2576 flags.go:64] FLAG: --register-with-taints=""
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623758    2576 flags.go:64] FLAG: --registry-burst="10"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623761    2576 flags.go:64] FLAG: --registry-qps="5"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623764    2576 flags.go:64] FLAG: --reserved-cpus=""
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623767    2576 flags.go:64] FLAG: --reserved-memory=""
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623771    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623774    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623777    2576 flags.go:64] FLAG: --rotate-certificates="false"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623779    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623782    2576 flags.go:64] FLAG: --runonce="false"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623785    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623788    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623791    2576 flags.go:64] FLAG: --seccomp-default="false"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623794    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623797    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623801    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623804    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623807    2576 flags.go:64] FLAG: --storage-driver-password="root"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623810    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623813    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623816    2576 flags.go:64] FLAG: --storage-driver-user="root"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623819    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 14 07:49:05.631618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623822    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623826    2576 flags.go:64] FLAG: --system-cgroups=""
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623829    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623834    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623837    2576 flags.go:64] FLAG: --tls-cert-file=""
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623840    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623843    2576 flags.go:64] FLAG: --tls-min-version=""
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623846    2576 flags.go:64] FLAG: --tls-private-key-file=""
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623849    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623852    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623855    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623858    2576 flags.go:64] FLAG: --v="2"
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623861    2576 flags.go:64] FLAG: --version="false"
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623865    2576 flags.go:64] FLAG: --vmodule=""
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623869    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.623872    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623985    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623989    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623992    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623995    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.623998    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624001    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624004    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:05.632304 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624007    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624010    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624014    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624016    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624019    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624022    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624025    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624028    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624032    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624035    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624037    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624041    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624043    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624046    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624050    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624054    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624057    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624059    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624062    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:05.632853 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624065    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624068    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624070    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624073    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624075    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624078    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624081    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624083    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624086    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624089    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624091    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624094    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624097    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624100    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624102    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624107    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624109    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624112    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624114    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624117    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:05.633335 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624120    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624124    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624127    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624129    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624132    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624135    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624138    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624140    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624143    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624146    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624148    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624151    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624154    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624156    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624159    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624163    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624167    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624170    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624173    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624176    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:05.633816 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624179    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624181    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624184    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624187    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624189    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624192    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624195    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624199    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624201    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624216    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624219    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624222    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624225    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624236    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624238    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624241    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624244    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624247    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624250    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624253    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:05.634312 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624256    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624259    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624263    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624266    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624269    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624272    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624275    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624278    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624280    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624283    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624285    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624288    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.624290    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.624299    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.631072    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 14 07:49:05.634802 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.631086    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.632588    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.632596    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632665    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632670    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632674    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632677    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632680    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632683    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632685    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632688    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632691    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632693    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632696    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632699    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632707    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632710    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632713    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632715    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632718    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:05.635178 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632721    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632724    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632727    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632729    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632732    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632736    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632740    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632743    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632746    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632749    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632751    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632754    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632758    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632762    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632765    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632767    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632771    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632773    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632776    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:05.635726 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632779    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632781    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632784    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632787    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632790    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632792    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632795    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632797    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632800    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632808    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632811    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632814    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632816    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632819    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632821    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632824    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632827    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632829    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632833    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632837    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:05.636191 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632840    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632842    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632845    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632847    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632850    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632853    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632856    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632859    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632863    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632867    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632870    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632873    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632875    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632878    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632881    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632884    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632887    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632890    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632892    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:05.636771 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632895    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632898    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632900    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632908    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632911    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632914    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632916    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632919    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632921    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632924    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632926    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632929    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632932    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632934    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632937    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632940    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632942    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632946    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632948    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:05.637246 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632951    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632954    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632956    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632959    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.632961    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.632966    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633115    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633120    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633122    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633125    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633128    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633131    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633133    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633136    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633139    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:05.637695 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633141    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633144    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633152    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633155    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633157    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633160    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633162    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633165    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633167    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633170    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633172    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633175    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633179    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633182    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633185    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633188    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633191    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633194    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633197    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633199    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:05.638059 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633202    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633221    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633225    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633228    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633230    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633233    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633235    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633238    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633240    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633243    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633245    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633248    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633250    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633253    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633255    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633264    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633267    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633270    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633273    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:05.638552 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633275    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633278    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633280    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633282    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633285    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633287    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633290    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633292    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633295    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633298    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633301    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633304    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633306    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633309    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633311    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633314    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633316    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633319    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633321    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633324    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:05.639033 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633326    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633329    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633333    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633337    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633340    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633345    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633348    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633351    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633353    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633363    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633366    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633368    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633371    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633373    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633376    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633378    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633381    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633383    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633386    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:05.639535 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633388    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633391    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633394    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633396    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633399    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633401    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633404    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633407    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633409    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633412    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633415    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:05.633417    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.633422    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.633529    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.635466    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 14 07:49:05.639991 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.636511    2576 server.go:1019] "Starting client certificate rotation"
Sep 14 07:49:05.640429 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.636610    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 07:49:05.640429 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.636646    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 07:49:05.640561 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.640548    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 07:49:05.643759 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.643739    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 07:49:05.658817 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.658799    2576 log.go:25] "Validated CRI v1 runtime API"
Sep 14 07:49:05.661926 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.661910    2576 log.go:25] "Validated CRI v1 image API"
Sep 14 07:49:05.662723 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.662709    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 14 07:49:05.664671 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.664648    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 07:49:05.666947 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.666925    2576 fs.go:135] Filesystem UUIDs: map[62fe1939-ed50-443f-bb42-a1a2ef944827:/dev/nvme0n1p3 665d3a7c-ecbe-46e6-a604-c9aa11df8de3:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 14 07:49:05.666999 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.666947    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 14 07:49:05.671855 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.671755    2576 manager.go:216] Machine: {Timestamp:2026-09-14 07:49:05.670480787 +0000 UTC m=+0.375019281 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3076276 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec204d1e6bb8724155e6449636996f17 SystemUUID:ec204d1e-6bb8-7241-55e6-449636996f17 BootID:6aece2ec-9b0d-4955-9331-4562e0cc9371 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d9:4b:03:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d9:4b:03:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:a9:f6:aa:ab:bf Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 14 07:49:05.671855 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.671849    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 14 07:49:05.671967 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.671914    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 14 07:49:05.673761 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.673740    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 14 07:49:05.673893 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.673764    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-123.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 14 07:49:05.674919 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.674908    2576 topology_manager.go:138] "Creating topology manager with none policy"
Sep 14 07:49:05.674919 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.674920    2576 container_manager_linux.go:309] "Creating device plugin manager"
Sep 14 07:49:05.675064 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.674928    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 07:49:05.675751 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.675737    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 07:49:05.676708 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.676695    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 14 07:49:05.680404 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.680391    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 07:49:05.680532 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.680523    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 14 07:49:05.680569 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.680553    2576 kubelet.go:486] "Attempting to sync node with API server"
Sep 14 07:49:05.680569 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.680565    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 14 07:49:05.680620 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.680578    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 14 07:49:05.680620 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.680587    2576 kubelet.go:398] "Adding apiserver pod source"
Sep 14 07:49:05.680620 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.680595    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 14 07:49:05.680937 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.680914    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 07:49:05.681030 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.680946    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 07:49:05.684114 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.684095    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 14 07:49:05.686692 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.686667    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 14 07:49:05.686760 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.686710    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 14 07:49:05.688577 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.688564    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 14 07:49:05.688577 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.688580    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 14 07:49:05.688692 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.688586    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 14 07:49:05.688692 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.688592    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 14 07:49:05.688692 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.688600    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 14 07:49:05.688692 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.688606    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 14 07:49:05.688692 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.688617    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 14 07:49:05.688692 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.688623    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 14 07:49:05.688692 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.688645    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 14 07:49:05.688692 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.688651    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 14 07:49:05.688692 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.688668    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 14 07:49:05.688692 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.688674    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 14 07:49:05.689338 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.689318    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 14 07:49:05.689338 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.689319    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 14 07:49:05.689494 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.689485    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 14 07:49:05.689494 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.689495    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 14 07:49:05.692864 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.692849    2576 server.go:1268] "Started kubelet"
Sep 14 07:49:05.692982 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.692941    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 14 07:49:05.693033 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.693003    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 14 07:49:05.693079 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.692973    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 14 07:49:05.695294 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.695268    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6w9b4"
Sep 14 07:49:05.695516 ip-10-0-133-123 systemd[1]: Started Kubernetes Kubelet.
Sep 14 07:49:05.695680 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.695665    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 14 07:49:05.697746 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.697080    2576 server.go:310] "Adding debug handlers to kubelet server"
Sep 14 07:49:05.699720 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.699517    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6w9b4"
Sep 14 07:49:05.701486 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.701464    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 14 07:49:05.701562 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.701485    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-123.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 14 07:49:05.702621 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.701550    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-123.ec2.internal.18d5207c6afec573  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-123.ec2.internal,UID:ip-10-0-133-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-123.ec2.internal,},FirstTimestamp:2026-09-14 07:49:05.692755315 +0000 UTC m=+0.397293808,LastTimestamp:2026-09-14 07:49:05.692755315 +0000 UTC m=+0.397293808,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-123.ec2.internal,}"
Sep 14 07:49:05.705101 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.705084    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 14 07:49:05.706714 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.706700    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 14 07:49:05.706815 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.706801    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:05.707467 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.707452    2576 factory.go:55] Registering systemd factory
Sep 14 07:49:05.707467 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.707467    2576 factory.go:223] Registration of the systemd container factory successfully
Sep 14 07:49:05.707604 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.707545    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 14 07:49:05.707604 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.707547    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 14 07:49:05.707604 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.707564    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 14 07:49:05.707715 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.707643    2576 reconstruct.go:97] "Volume reconstruction finished"
Sep 14 07:49:05.707715 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.707650    2576 reconciler.go:29] "Reconciler: start to sync state"
Sep 14 07:49:05.707715 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.707674    2576 factory.go:153] Registering CRI-O factory
Sep 14 07:49:05.707715 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.707687    2576 factory.go:223] Registration of the crio container factory successfully
Sep 14 07:49:05.707899 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.707735    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 14 07:49:05.707899 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.707751    2576 factory.go:103] Registering Raw factory
Sep 14 07:49:05.707899 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.707768    2576 manager.go:1196] Started watching for new ooms in manager
Sep 14 07:49:05.708045 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.708031    2576 manager.go:349] Starting recovery of all containers
Sep 14 07:49:05.708816 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.708797    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:05.711187 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.711169    2576 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-123.ec2.internal\" not found" node="ip-10-0-133-123.ec2.internal"
Sep 14 07:49:05.717407 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.717392    2576 manager.go:354] Recovery completed
Sep 14 07:49:05.722677 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.722591    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:05.724238 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.724196    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:05.724297 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.724244    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:05.724297 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.724255    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:05.724748 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.724736    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 14 07:49:05.724812 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.724748    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 14 07:49:05.724812 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.724766    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:05.724812 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.724775    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 07:49:05.725426 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.725394    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:05.725426 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.725416    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:05.725426 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.725424    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:05.726046 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.726035    2576 policy_none.go:49] "None policy: Start"
Sep 14 07:49:05.726091 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.726051    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 14 07:49:05.726091 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.726060    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 14 07:49:05.727135 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.727124    2576 policy_none.go:47] "Start"
Sep 14 07:49:05.774301 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.756859    2576 manager.go:337] "Starting Device Plugin manager"
Sep 14 07:49:05.774301 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.756888    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 14 07:49:05.774301 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.756897    2576 server.go:87] "Starting device plugin registration server"
Sep 14 07:49:05.774301 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.757114    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 14 07:49:05.774301 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.757123    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 14 07:49:05.774301 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.757195    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 14 07:49:05.774301 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.757286    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 14 07:49:05.774301 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.757297    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 14 07:49:05.774301 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.757921    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 14 07:49:05.774301 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.757971    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:05.830718 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.830697    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 14 07:49:05.831824 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.831805    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 14 07:49:05.831898 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.831853    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 14 07:49:05.831898 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.831872    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 14 07:49:05.831978 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.831937    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 14 07:49:05.834399 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.834378    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:05.857437 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.857423    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:05.858182 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.858167    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:05.858278 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.858197    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:05.858278 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.858224    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:05.858278 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.858253    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-123.ec2.internal"
Sep 14 07:49:05.866994 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.866946    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-123.ec2.internal"
Sep 14 07:49:05.866994 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.866965    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-123.ec2.internal\": node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:05.894109 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.894089    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:05.933845 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.933823    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal"]
Sep 14 07:49:05.933936 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.933887    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:05.934690 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.934672    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:05.934767 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.934703    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:05.934767 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.934716    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:05.936019 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.936007    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:05.936621 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.936605    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:05.936750 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.936637    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:05.936750 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.936661    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:05.936750 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.936680    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:05.936750 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.936695    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:05.937266 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.937253    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:05.937331 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.937276    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:05.937331 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.937285    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:05.937790 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.937775    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:05.937859 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.937803    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:05.938507 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.938492    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:05.938596 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.938522    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:05.938596 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:05.938535    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:05.961828 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.961807    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-123.ec2.internal\" not found" node="ip-10-0-133-123.ec2.internal"
Sep 14 07:49:05.966091 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.966076    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-123.ec2.internal\" not found" node="ip-10-0-133-123.ec2.internal"
Sep 14 07:49:05.995039 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:05.995022    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:06.008888 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.008866    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8c553a2f2a397bf530bccc9a54adec3a-config\") pod \"kube-apiserver-proxy-ip-10-0-133-123.ec2.internal\" (UID: \"8c553a2f2a397bf530bccc9a54adec3a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:06.008960 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.008935    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cca7b62a5f07be9c100b06a15efd970c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal\" (UID: \"cca7b62a5f07be9c100b06a15efd970c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:06.009002 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.008973    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca7b62a5f07be9c100b06a15efd970c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal\" (UID: \"cca7b62a5f07be9c100b06a15efd970c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:06.095903 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:06.095880    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:06.109287 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.109196    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca7b62a5f07be9c100b06a15efd970c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal\" (UID: \"cca7b62a5f07be9c100b06a15efd970c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:06.109342 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.109270    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca7b62a5f07be9c100b06a15efd970c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal\" (UID: \"cca7b62a5f07be9c100b06a15efd970c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:06.109342 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.109306    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8c553a2f2a397bf530bccc9a54adec3a-config\") pod \"kube-apiserver-proxy-ip-10-0-133-123.ec2.internal\" (UID: \"8c553a2f2a397bf530bccc9a54adec3a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:06.109342 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.109328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cca7b62a5f07be9c100b06a15efd970c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal\" (UID: \"cca7b62a5f07be9c100b06a15efd970c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:06.109438 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.109353    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cca7b62a5f07be9c100b06a15efd970c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal\" (UID: \"cca7b62a5f07be9c100b06a15efd970c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:06.109438 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.109388    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8c553a2f2a397bf530bccc9a54adec3a-config\") pod \"kube-apiserver-proxy-ip-10-0-133-123.ec2.internal\" (UID: \"8c553a2f2a397bf530bccc9a54adec3a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:06.196638 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:06.196595    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:06.265010 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.264995    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:06.268833 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.268820    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:06.297078 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:06.297057    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:06.400828 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:06.400794    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:06.501226 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:06.501166    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:06.562356 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.562329    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:06.602589 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:06.602551    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:06.637371 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.637342    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 14 07:49:06.637828 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.637465    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 07:49:06.637828 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.637522    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 07:49:06.637828 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.637533    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 07:49:06.702723 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:06.702699    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:06.705075 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.705042    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-13 07:44:06 +0000 UTC" deadline="2028-04-06 19:06:51.523447026 +0000 UTC"
Sep 14 07:49:06.705153 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.705077    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13691h17m44.818374521s"
Sep 14 07:49:06.803620 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:06.803568    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:06.829013 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:06.828987    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c553a2f2a397bf530bccc9a54adec3a.slice/crio-7f6faae76e2416844529195e1d1de532c48ffda12bcfd56e16fd65c315fb5113 WatchSource:0}: Error finding container 7f6faae76e2416844529195e1d1de532c48ffda12bcfd56e16fd65c315fb5113: Status 404 returned error can't find the container with id 7f6faae76e2416844529195e1d1de532c48ffda12bcfd56e16fd65c315fb5113
Sep 14 07:49:06.829489 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:06.829468    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcca7b62a5f07be9c100b06a15efd970c.slice/crio-dfebecccd4af24888eb83e0a43640869210b9a1571d1174765a20179757f8f65 WatchSource:0}: Error finding container dfebecccd4af24888eb83e0a43640869210b9a1571d1174765a20179757f8f65: Status 404 returned error can't find the container with id dfebecccd4af24888eb83e0a43640869210b9a1571d1174765a20179757f8f65
Sep 14 07:49:06.834244 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.834213    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:49:06.835236 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.835188    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal" event={"ID":"8c553a2f2a397bf530bccc9a54adec3a","Type":"ContainerStarted","Data":"7f6faae76e2416844529195e1d1de532c48ffda12bcfd56e16fd65c315fb5113"}
Sep 14 07:49:06.836512 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:06.836489    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" event={"ID":"cca7b62a5f07be9c100b06a15efd970c","Type":"ContainerStarted","Data":"dfebecccd4af24888eb83e0a43640869210b9a1571d1174765a20179757f8f65"}
Sep 14 07:49:06.904151 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:06.904127    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:07.004521 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:07.004493    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:07.104599 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:07.104571    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:07.205013 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:07.204968    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found"
Sep 14 07:49:07.226263 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.226236    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:07.306983 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.306963    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:07.319995 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.319967    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 07:49:07.321078 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.321033    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal"
Sep 14 07:49:07.327675 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.327653    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 07:49:07.586554 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.586494    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:07.682203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.682177    2576 apiserver.go:52] "Watching apiserver"
Sep 14 07:49:07.690411 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.690384    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 14 07:49:07.690871 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.690845    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-j9vrm","openshift-ovn-kubernetes/ovnkube-node-x6n47","kube-system/global-pull-secret-syncer-fl64c","kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm","openshift-dns/node-resolver-vqbl2","openshift-multus/multus-additional-cni-plugins-7zkjj","openshift-multus/multus-pwcsr","openshift-multus/network-metrics-daemon-pwczh","openshift-network-diagnostics/network-check-target-qjv9w","kube-system/konnectivity-agent-6bfdg","openshift-cluster-node-tuning-operator/tuned-5xps5","openshift-image-registry/node-ca-qgb6w","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal"]
Sep 14 07:49:07.693080 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.693054    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:07.693186 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:07.693143    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:07.694668 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.694641    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qgb6w"
Sep 14 07:49:07.695917 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.695893    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:07.695998 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:07.695965    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:07.697793 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.697586    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 14 07:49:07.697793 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.697638    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 14 07:49:07.697793 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.697649    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 14 07:49:07.697984 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.697809    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bxgjx\""
Sep 14 07:49:07.698518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.698494    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.700799 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.700779    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 14 07:49:07.701052 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.701034    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 14 07:49:07.701171 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.701152    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f4w6c\""
Sep 14 07:49:07.701235 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.701038    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 14 07:49:07.701311 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.700780    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 14 07:49:07.701420 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.701396    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.702859 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.702838    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vqbl2"
Sep 14 07:49:07.703860 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.703840    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 14 07:49:07.703965 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.703872    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 14 07:49:07.704189 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.704169    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wvkx4\""
Sep 14 07:49:07.704433 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.704415    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 14 07:49:07.704542 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.704477    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:07.704598 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:07.704537    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:07.705726 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.705275    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 14 07:49:07.705726 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.705287    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wgqx9\""
Sep 14 07:49:07.705726 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.705433    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 14 07:49:07.706110 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.706087    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 14 07:49:07.706600 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.706571    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.708122 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.708101    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.709106 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.709086    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 14 07:49:07.709191 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.709160    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 14 07:49:07.709312 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.709293    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sxsxh\""
Sep 14 07:49:07.710145 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.710121    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j9vrm"
Sep 14 07:49:07.710606 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.710585    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 14 07:49:07.710709 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.710633    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k96cq\""
Sep 14 07:49:07.710709 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.710647    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 14 07:49:07.710831 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.710714    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 14 07:49:07.711515 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.711493    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 14 07:49:07.712503 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.712481    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 14 07:49:07.712682 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.712663    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6bfdg"
Sep 14 07:49:07.712767 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.712752    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:49:07.712860 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.712843    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 14 07:49:07.713148 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.713125    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-p5vq8\""
Sep 14 07:49:07.713264 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.713210    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 14 07:49:07.713836 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.713812    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 14 07:49:07.715737 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.715709    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fr5zv\""
Sep 14 07:49:07.715737 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.715729    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.716080 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.716059    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 14 07:49:07.716644 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.716625    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 14 07:49:07.716957 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.716936    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-socket-dir\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.717026 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.716977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd19124b-82d8-4f39-a028-cfc81c6979c8-hosts-file\") pod \"node-resolver-vqbl2\" (UID: \"bd19124b-82d8-4f39-a028-cfc81c6979c8\") " pod="openshift-dns/node-resolver-vqbl2"
Sep 14 07:49:07.717026 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717014    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41a95c54-5096-456d-843c-62ee949876c2-host\") pod \"node-ca-qgb6w\" (UID: \"41a95c54-5096-456d-843c-62ee949876c2\") " pod="openshift-image-registry/node-ca-qgb6w"
Sep 14 07:49:07.717137 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717055    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-cnibin\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.717137 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717120    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-etc-selinux\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.717233 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717151    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/41a95c54-5096-456d-843c-62ee949876c2-serviceca\") pod \"node-ca-qgb6w\" (UID: \"41a95c54-5096-456d-843c-62ee949876c2\") " pod="openshift-image-registry/node-ca-qgb6w"
Sep 14 07:49:07.717233 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717202    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-etc-kubernetes\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.717352 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717235    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-var-lib-openvswitch\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.717352 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717266    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-run-openvswitch\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.717449 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717377    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-cni-bin\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.717499 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717479    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:07.717549 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717519    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-multus-cni-dir\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.717603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717558    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c850c672-bf8a-4935-830c-db4cd90d3c6b-cni-binary-copy\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.717653 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717610    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-log-socket\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.717705 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717651    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-run-ovn-kubernetes\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.717705 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717677    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c850c672-bf8a-4935-830c-db4cd90d3c6b-system-cni-dir\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.717799 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717725    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvzzj\" (UniqueName: \"kubernetes.io/projected/c850c672-bf8a-4935-830c-db4cd90d3c6b-kube-api-access-zvzzj\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.717799 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717756    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lxjh\" (UniqueName: \"kubernetes.io/projected/41a95c54-5096-456d-843c-62ee949876c2-kube-api-access-9lxjh\") pod \"node-ca-qgb6w\" (UID: \"41a95c54-5096-456d-843c-62ee949876c2\") " pod="openshift-image-registry/node-ca-qgb6w"
Sep 14 07:49:07.717799 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717780    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-cni-binary-copy\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.717936 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717810    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-multus-socket-dir-parent\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.717936 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717858    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs\") pod \"network-metrics-daemon-pwczh\" (UID: \"89ce251d-66b1-4990-8d8a-101dafeb009b\") " pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:07.717936 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717888    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c850c672-bf8a-4935-830c-db4cd90d3c6b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.717936 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717918    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-run-k8s-cni-cncf-io\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.718120 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717942    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-multus-daemon-config\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.718120 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.717972    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-run-netns\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.718120 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718002    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szqvn\" (UniqueName: \"kubernetes.io/projected/9e1221aa-49ae-48fa-819a-f7d296b01557-kube-api-access-szqvn\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.718120 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718030    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-run-netns\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.718120 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-run-multus-certs\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.718120 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718094    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:49:07.718429 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718132    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-sys-fs\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.718429 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718173    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9e1221aa-49ae-48fa-819a-f7d296b01557-env-overrides\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.718429 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718204    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9e1221aa-49ae-48fa-819a-f7d296b01557-ovnkube-script-lib\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.718429 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718231    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9wkm\" (UniqueName: \"kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm\") pod \"network-check-target-qjv9w\" (UID: \"67bb572c-7609-4a7c-99ab-fcb047a0fc7b\") " pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:07.718429 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-system-cni-dir\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.718429 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718314    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-var-lib-cni-bin\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.718704 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718428    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-multus-conf-dir\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.718704 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718491    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr4lv\" (UniqueName: \"kubernetes.io/projected/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-kube-api-access-fr4lv\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.718704 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718542    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fmnb\" (UniqueName: \"kubernetes.io/projected/3d441338-3dfd-4b6a-a535-f7af399bfe70-kube-api-access-9fmnb\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.718704 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-slash\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.718704 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718629    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.718704 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718664    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2flww\" (UniqueName: \"kubernetes.io/projected/bd19124b-82d8-4f39-a028-cfc81c6979c8-kube-api-access-2flww\") pod \"node-resolver-vqbl2\" (UID: \"bd19124b-82d8-4f39-a028-cfc81c6979c8\") " pod="openshift-dns/node-resolver-vqbl2"
Sep 14 07:49:07.718704 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718667    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hf4cz\""
Sep 14 07:49:07.719080 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718667    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 14 07:49:07.719080 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718743    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-var-lib-kubelet\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.719080 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718820    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd19124b-82d8-4f39-a028-cfc81c6979c8-tmp-dir\") pod \"node-resolver-vqbl2\" (UID: \"bd19124b-82d8-4f39-a028-cfc81c6979c8\") " pod="openshift-dns/node-resolver-vqbl2"
Sep 14 07:49:07.719080 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718863    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c850c672-bf8a-4935-830c-db4cd90d3c6b-cnibin\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.719080 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718897    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-hostroot\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.719080 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.718970    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-kubelet\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.719406 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719116    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-run-ovn\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.719406 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.719406 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719234    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-device-dir\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.719406 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-etc-openvswitch\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.719406 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719279    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-node-log\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.719406 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719308    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c850c672-bf8a-4935-830c-db4cd90d3c6b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.719406 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-systemd-units\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.719406 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719393    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-cni-netd\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.719823 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719426    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5271f263-1387-45f8-9d53-db6e1b9df066-kubelet-config\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:07.719823 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719426    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 07:49:07.719823 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719488    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5271f263-1387-45f8-9d53-db6e1b9df066-dbus\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:07.719823 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719524    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c850c672-bf8a-4935-830c-db4cd90d3c6b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.719823 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719562    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-registration-dir\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.719823 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719600    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpqc6\" (UniqueName: \"kubernetes.io/projected/89ce251d-66b1-4990-8d8a-101dafeb009b-kube-api-access-kpqc6\") pod \"network-metrics-daemon-pwczh\" (UID: \"89ce251d-66b1-4990-8d8a-101dafeb009b\") " pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:07.719823 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.719634    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-run-systemd\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.720227 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.720140    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9e1221aa-49ae-48fa-819a-f7d296b01557-ovnkube-config\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.720227 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.720173    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-var-lib-cni-multus\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.720227 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.720208    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e1221aa-49ae-48fa-819a-f7d296b01557-ovn-node-metrics-cert\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.742004 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.741981    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-z6j4n"
Sep 14 07:49:07.748624 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.748601    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-z6j4n"
Sep 14 07:49:07.808650 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.808627    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 14 07:49:07.821411 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.821383    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-cni-binary-copy\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.821514 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.821424    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-multus-socket-dir-parent\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.821514 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.821458    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be9c0768-7db5-401c-a564-6f06859a183d-iptables-alerter-script\") pod \"iptables-alerter-j9vrm\" (UID: \"be9c0768-7db5-401c-a564-6f06859a183d\") " pod="openshift-network-operator/iptables-alerter-j9vrm"
Sep 14 07:49:07.821514 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.821481    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-sysctl-d\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.821647 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.821515    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-multus-socket-dir-parent\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.821647 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.821532    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs\") pod \"network-metrics-daemon-pwczh\" (UID: \"89ce251d-66b1-4990-8d8a-101dafeb009b\") " pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:07.821647 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.821564    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c850c672-bf8a-4935-830c-db4cd90d3c6b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.821647 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:07.821628    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:07.821848 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.821670    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-lib-modules\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.821848 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:07.821772    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs podName:89ce251d-66b1-4990-8d8a-101dafeb009b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:08.321722078 +0000 UTC m=+3.026260584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs") pod "network-metrics-daemon-pwczh" (UID: "89ce251d-66b1-4990-8d8a-101dafeb009b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:07.821848 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.821816    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-run-k8s-cni-cncf-io\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822013 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.821850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-multus-daemon-config\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822013 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.821896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-run-netns\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.822013 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.821893    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-run-k8s-cni-cncf-io\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822013 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.821923    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szqvn\" (UniqueName: \"kubernetes.io/projected/9e1221aa-49ae-48fa-819a-f7d296b01557-kube-api-access-szqvn\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.822214 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822019    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-cni-binary-copy\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822214 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822045    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-run-netns\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822214 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822062    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-run-netns\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.822214 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822095    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-run-multus-certs\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822214 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822116    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-run-netns\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822214 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822123    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-sys-fs\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.822214 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-run-multus-certs\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822214 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822161    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9e1221aa-49ae-48fa-819a-f7d296b01557-env-overrides\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.822214 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822175    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c850c672-bf8a-4935-830c-db4cd90d3c6b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.822214 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822196    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9e1221aa-49ae-48fa-819a-f7d296b01557-ovnkube-script-lib\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822225    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9wkm\" (UniqueName: \"kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm\") pod \"network-check-target-qjv9w\" (UID: \"67bb572c-7609-4a7c-99ab-fcb047a0fc7b\") " pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822248    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-sys-fs\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822253    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-sys\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822306    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-system-cni-dir\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-var-lib-cni-bin\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822362    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-multus-conf-dir\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822390    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr4lv\" (UniqueName: \"kubernetes.io/projected/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-kube-api-access-fr4lv\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fmnb\" (UniqueName: \"kubernetes.io/projected/3d441338-3dfd-4b6a-a535-f7af399bfe70-kube-api-access-9fmnb\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822456    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-system-cni-dir\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822445    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-slash\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822467    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-multus-daemon-config\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822512    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822519    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-slash\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822552    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2flww\" (UniqueName: \"kubernetes.io/projected/bd19124b-82d8-4f39-a028-cfc81c6979c8-kube-api-access-2flww\") pod \"node-resolver-vqbl2\" (UID: \"bd19124b-82d8-4f39-a028-cfc81c6979c8\") " pod="openshift-dns/node-resolver-vqbl2"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822559    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822580    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-var-lib-kubelet\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.822740 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822585    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-var-lib-cni-bin\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822605    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd19124b-82d8-4f39-a028-cfc81c6979c8-tmp-dir\") pod \"node-resolver-vqbl2\" (UID: \"bd19124b-82d8-4f39-a028-cfc81c6979c8\") " pod="openshift-dns/node-resolver-vqbl2"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822622    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9e1221aa-49ae-48fa-819a-f7d296b01557-env-overrides\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822632    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c850c672-bf8a-4935-830c-db4cd90d3c6b-cnibin\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822660    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-var-lib-kubelet\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822687    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-host\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-hostroot\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822721    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9e1221aa-49ae-48fa-819a-f7d296b01557-ovnkube-script-lib\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822737    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-kubelet\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822764    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-run-ovn\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822551    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-multus-conf-dir\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822790    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c850c672-bf8a-4935-830c-db4cd90d3c6b-cnibin\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822804    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-modprobe-d\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822822    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-hostroot\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822831    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-kubelet\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822837    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4spl\" (UniqueName: \"kubernetes.io/projected/08f5e8f4-1fcd-4186-a367-3697f02565ca-kube-api-access-m4spl\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822873    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-var-lib-kubelet\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822872    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.823628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822895    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-run-ovn\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822906    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-device-dir\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822925    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822924    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-etc-openvswitch\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822951    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-node-log\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822967    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c850c672-bf8a-4935-830c-db4cd90d3c6b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.822996    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be9c0768-7db5-401c-a564-6f06859a183d-host-slash\") pod \"iptables-alerter-j9vrm\" (UID: \"be9c0768-7db5-401c-a564-6f06859a183d\") " pod="openshift-network-operator/iptables-alerter-j9vrm"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823004    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd19124b-82d8-4f39-a028-cfc81c6979c8-tmp-dir\") pod \"node-resolver-vqbl2\" (UID: \"bd19124b-82d8-4f39-a028-cfc81c6979c8\") " pod="openshift-dns/node-resolver-vqbl2"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823040    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-run\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-systemd-units\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823078    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-etc-openvswitch\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823042    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-device-dir\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823101    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-cni-netd\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823113    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-systemd-units\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5271f263-1387-45f8-9d53-db6e1b9df066-kubelet-config\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823069    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-node-log\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-cni-netd\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.824518 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5271f263-1387-45f8-9d53-db6e1b9df066-dbus\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823178    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c850c672-bf8a-4935-830c-db4cd90d3c6b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c850c672-bf8a-4935-830c-db4cd90d3c6b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5271f263-1387-45f8-9d53-db6e1b9df066-dbus\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823212    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-kubernetes\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823180    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5271f263-1387-45f8-9d53-db6e1b9df066-kubelet-config\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823253    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-tuned\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823314    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-registration-dir\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpqc6\" (UniqueName: \"kubernetes.io/projected/89ce251d-66b1-4990-8d8a-101dafeb009b-kube-api-access-kpqc6\") pod \"network-metrics-daemon-pwczh\" (UID: \"89ce251d-66b1-4990-8d8a-101dafeb009b\") " pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823370    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-run-systemd\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823394    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9e1221aa-49ae-48fa-819a-f7d296b01557-ovnkube-config\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823423    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-sysconfig\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823425    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-run-systemd\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823451    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-var-lib-cni-multus\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-registration-dir\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e1221aa-49ae-48fa-819a-f7d296b01557-ovn-node-metrics-cert\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823507    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41-konnectivity-ca\") pod \"konnectivity-agent-6bfdg\" (UID: \"55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41\") " pod="kube-system/konnectivity-agent-6bfdg"
Sep 14 07:49:07.825245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823511    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-host-var-lib-cni-multus\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-socket-dir\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd19124b-82d8-4f39-a028-cfc81c6979c8-hosts-file\") pod \"node-resolver-vqbl2\" (UID: \"bd19124b-82d8-4f39-a028-cfc81c6979c8\") " pod="openshift-dns/node-resolver-vqbl2"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823658    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08f5e8f4-1fcd-4186-a367-3697f02565ca-tmp\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823670    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c850c672-bf8a-4935-830c-db4cd90d3c6b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823659    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-socket-dir\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823685    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41a95c54-5096-456d-843c-62ee949876c2-host\") pod \"node-ca-qgb6w\" (UID: \"41a95c54-5096-456d-843c-62ee949876c2\") " pod="openshift-image-registry/node-ca-qgb6w"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd19124b-82d8-4f39-a028-cfc81c6979c8-hosts-file\") pod \"node-resolver-vqbl2\" (UID: \"bd19124b-82d8-4f39-a028-cfc81c6979c8\") " pod="openshift-dns/node-resolver-vqbl2"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-cnibin\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823752    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-etc-selinux\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823725    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41a95c54-5096-456d-843c-62ee949876c2-host\") pod \"node-ca-qgb6w\" (UID: \"41a95c54-5096-456d-843c-62ee949876c2\") " pod="openshift-image-registry/node-ca-qgb6w"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/41a95c54-5096-456d-843c-62ee949876c2-serviceca\") pod \"node-ca-qgb6w\" (UID: \"41a95c54-5096-456d-843c-62ee949876c2\") " pod="openshift-image-registry/node-ca-qgb6w"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823768    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-cnibin\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823802    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3d441338-3dfd-4b6a-a535-f7af399bfe70-etc-selinux\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823833    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9e1221aa-49ae-48fa-819a-f7d296b01557-ovnkube-config\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823806    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-etc-kubernetes\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823861    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-etc-kubernetes\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823879    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-var-lib-openvswitch\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.825884 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823903    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-run-openvswitch\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823928    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-cni-bin\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823953    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823944    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-var-lib-openvswitch\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823963    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823990    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-run-openvswitch\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.823988    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hmcr\" (UniqueName: \"kubernetes.io/projected/be9c0768-7db5-401c-a564-6f06859a183d-kube-api-access-6hmcr\") pod \"iptables-alerter-j9vrm\" (UID: \"be9c0768-7db5-401c-a564-6f06859a183d\") " pod="openshift-network-operator/iptables-alerter-j9vrm"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824021    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-cni-bin\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824037    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-sysctl-conf\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-multus-cni-dir\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:07.824078    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824091    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c850c672-bf8a-4935-830c-db4cd90d3c6b-cni-binary-copy\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:07.824125    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret podName:5271f263-1387-45f8-9d53-db6e1b9df066 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:08.324108637 +0000 UTC m=+3.028647137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret") pod "global-pull-secret-syncer-fl64c" (UID: "5271f263-1387-45f8-9d53-db6e1b9df066") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824142    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-systemd\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824167    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41-agent-certs\") pod \"konnectivity-agent-6bfdg\" (UID: \"55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41\") " pod="kube-system/konnectivity-agent-6bfdg"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824174    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-multus-cni-dir\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824194    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-log-socket\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.826603 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/41a95c54-5096-456d-843c-62ee949876c2-serviceca\") pod \"node-ca-qgb6w\" (UID: \"41a95c54-5096-456d-843c-62ee949876c2\") " pod="openshift-image-registry/node-ca-qgb6w"
Sep 14 07:49:07.827217 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824222    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-run-ovn-kubernetes\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.827217 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824251    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c850c672-bf8a-4935-830c-db4cd90d3c6b-system-cni-dir\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.827217 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824256    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-log-socket\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.827217 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824298    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvzzj\" (UniqueName: \"kubernetes.io/projected/c850c672-bf8a-4935-830c-db4cd90d3c6b-kube-api-access-zvzzj\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.827217 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824307    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e1221aa-49ae-48fa-819a-f7d296b01557-host-run-ovn-kubernetes\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.827217 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824311    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c850c672-bf8a-4935-830c-db4cd90d3c6b-system-cni-dir\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.827217 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824343    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lxjh\" (UniqueName: \"kubernetes.io/projected/41a95c54-5096-456d-843c-62ee949876c2-kube-api-access-9lxjh\") pod \"node-ca-qgb6w\" (UID: \"41a95c54-5096-456d-843c-62ee949876c2\") " pod="openshift-image-registry/node-ca-qgb6w"
Sep 14 07:49:07.827217 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.824563    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c850c672-bf8a-4935-830c-db4cd90d3c6b-cni-binary-copy\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.828214 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.828196    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e1221aa-49ae-48fa-819a-f7d296b01557-ovn-node-metrics-cert\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.832273 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:07.832070    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:07.832273 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:07.832090    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:07.832273 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:07.832104    2576 projected.go:196] Error preparing data for projected volume kube-api-access-t9wkm for pod openshift-network-diagnostics/network-check-target-qjv9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:07.832273 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:07.832167    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm podName:67bb572c-7609-4a7c-99ab-fcb047a0fc7b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:08.332150301 +0000 UTC m=+3.036688790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-t9wkm" (UniqueName: "kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm") pod "network-check-target-qjv9w" (UID: "67bb572c-7609-4a7c-99ab-fcb047a0fc7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:07.834818 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.834472    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fmnb\" (UniqueName: \"kubernetes.io/projected/3d441338-3dfd-4b6a-a535-f7af399bfe70-kube-api-access-9fmnb\") pod \"aws-ebs-csi-driver-node-zkhdm\" (UID: \"3d441338-3dfd-4b6a-a535-f7af399bfe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:07.834965 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.834943    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szqvn\" (UniqueName: \"kubernetes.io/projected/9e1221aa-49ae-48fa-819a-f7d296b01557-kube-api-access-szqvn\") pod \"ovnkube-node-x6n47\" (UID: \"9e1221aa-49ae-48fa-819a-f7d296b01557\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:07.835401 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.835361    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr4lv\" (UniqueName: \"kubernetes.io/projected/505f1fe1-8e14-4b98-b9b8-9a90faf97b5d-kube-api-access-fr4lv\") pod \"multus-pwcsr\" (UID: \"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d\") " pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:07.836108 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.836035    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvzzj\" (UniqueName: \"kubernetes.io/projected/c850c672-bf8a-4935-830c-db4cd90d3c6b-kube-api-access-zvzzj\") pod \"multus-additional-cni-plugins-7zkjj\" (UID: \"c850c672-bf8a-4935-830c-db4cd90d3c6b\") " pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:07.836523 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.836503    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2flww\" (UniqueName: \"kubernetes.io/projected/bd19124b-82d8-4f39-a028-cfc81c6979c8-kube-api-access-2flww\") pod \"node-resolver-vqbl2\" (UID: \"bd19124b-82d8-4f39-a028-cfc81c6979c8\") " pod="openshift-dns/node-resolver-vqbl2"
Sep 14 07:49:07.836523 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.836521    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lxjh\" (UniqueName: \"kubernetes.io/projected/41a95c54-5096-456d-843c-62ee949876c2-kube-api-access-9lxjh\") pod \"node-ca-qgb6w\" (UID: \"41a95c54-5096-456d-843c-62ee949876c2\") " pod="openshift-image-registry/node-ca-qgb6w"
Sep 14 07:49:07.837666 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.837617    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpqc6\" (UniqueName: \"kubernetes.io/projected/89ce251d-66b1-4990-8d8a-101dafeb009b-kube-api-access-kpqc6\") pod \"network-metrics-daemon-pwczh\" (UID: \"89ce251d-66b1-4990-8d8a-101dafeb009b\") " pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:07.925086 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925058    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hmcr\" (UniqueName: \"kubernetes.io/projected/be9c0768-7db5-401c-a564-6f06859a183d-kube-api-access-6hmcr\") pod \"iptables-alerter-j9vrm\" (UID: \"be9c0768-7db5-401c-a564-6f06859a183d\") " pod="openshift-network-operator/iptables-alerter-j9vrm"
Sep 14 07:49:07.925086 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925086    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-sysctl-conf\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925267 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925109    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-systemd\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925267 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925129    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41-agent-certs\") pod \"konnectivity-agent-6bfdg\" (UID: \"55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41\") " pod="kube-system/konnectivity-agent-6bfdg"
Sep 14 07:49:07.925267 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925157    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be9c0768-7db5-401c-a564-6f06859a183d-iptables-alerter-script\") pod \"iptables-alerter-j9vrm\" (UID: \"be9c0768-7db5-401c-a564-6f06859a183d\") " pod="openshift-network-operator/iptables-alerter-j9vrm"
Sep 14 07:49:07.925267 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925221    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-systemd\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925452 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925267    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-sysctl-conf\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925452 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925311    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-sysctl-d\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925452 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925352    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-lib-modules\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925452 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925397    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-sys\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925452 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925432    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-var-lib-kubelet\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925670 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925456    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-host\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925670 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925483    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-modprobe-d\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925670 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925507    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4spl\" (UniqueName: \"kubernetes.io/projected/08f5e8f4-1fcd-4186-a367-3697f02565ca-kube-api-access-m4spl\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925670 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be9c0768-7db5-401c-a564-6f06859a183d-host-slash\") pod \"iptables-alerter-j9vrm\" (UID: \"be9c0768-7db5-401c-a564-6f06859a183d\") " pod="openshift-network-operator/iptables-alerter-j9vrm"
Sep 14 07:49:07.925670 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925543    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-lib-modules\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925670 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925554    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-run\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925670 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925581    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-sysctl-d\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925670 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925597    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-run\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925670 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-kubernetes\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925670 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925633    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-tuned\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925670 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925655    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-kubernetes\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.925670 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925663    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-sysconfig\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.926215 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925693    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41-konnectivity-ca\") pod \"konnectivity-agent-6bfdg\" (UID: \"55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41\") " pod="kube-system/konnectivity-agent-6bfdg"
Sep 14 07:49:07.926215 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925698    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-host\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.926215 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925717    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be9c0768-7db5-401c-a564-6f06859a183d-iptables-alerter-script\") pod \"iptables-alerter-j9vrm\" (UID: \"be9c0768-7db5-401c-a564-6f06859a183d\") " pod="openshift-network-operator/iptables-alerter-j9vrm"
Sep 14 07:49:07.926215 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925732    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-var-lib-kubelet\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.926215 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925721    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08f5e8f4-1fcd-4186-a367-3697f02565ca-tmp\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.926215 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925693    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-modprobe-d\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.926215 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925740    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-sys\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.926215 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925813    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be9c0768-7db5-401c-a564-6f06859a183d-host-slash\") pod \"iptables-alerter-j9vrm\" (UID: \"be9c0768-7db5-401c-a564-6f06859a183d\") " pod="openshift-network-operator/iptables-alerter-j9vrm"
Sep 14 07:49:07.926215 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.925997    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-sysconfig\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.926727 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.926404    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41-konnectivity-ca\") pod \"konnectivity-agent-6bfdg\" (UID: \"55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41\") " pod="kube-system/konnectivity-agent-6bfdg"
Sep 14 07:49:07.927958 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.927931    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08f5e8f4-1fcd-4186-a367-3697f02565ca-tmp\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.928111 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.928095    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08f5e8f4-1fcd-4186-a367-3697f02565ca-etc-tuned\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.928341 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.928319    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41-agent-certs\") pod \"konnectivity-agent-6bfdg\" (UID: \"55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41\") " pod="kube-system/konnectivity-agent-6bfdg"
Sep 14 07:49:07.934075 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.934055    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4spl\" (UniqueName: \"kubernetes.io/projected/08f5e8f4-1fcd-4186-a367-3697f02565ca-kube-api-access-m4spl\") pod \"tuned-5xps5\" (UID: \"08f5e8f4-1fcd-4186-a367-3697f02565ca\") " pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:07.934465 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.934444    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hmcr\" (UniqueName: \"kubernetes.io/projected/be9c0768-7db5-401c-a564-6f06859a183d-kube-api-access-6hmcr\") pod \"iptables-alerter-j9vrm\" (UID: \"be9c0768-7db5-401c-a564-6f06859a183d\") " pod="openshift-network-operator/iptables-alerter-j9vrm"
Sep 14 07:49:07.957877 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:07.957857    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:08.016656 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.016632    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qgb6w"
Sep 14 07:49:08.025054 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.025032    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pwcsr"
Sep 14 07:49:08.036273 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.036251    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm"
Sep 14 07:49:08.041742 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.041722    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vqbl2"
Sep 14 07:49:08.050127 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.050106    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7zkjj"
Sep 14 07:49:08.057410 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.057392    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:08.065713 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.065694    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j9vrm"
Sep 14 07:49:08.073438 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.073420    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6bfdg"
Sep 14 07:49:08.080955 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.080932    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5xps5"
Sep 14 07:49:08.328766 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.328733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs\") pod \"network-metrics-daemon-pwczh\" (UID: \"89ce251d-66b1-4990-8d8a-101dafeb009b\") " pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:08.328939 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.328811    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:08.328939 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:08.328909    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:08.329067 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:08.328976    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs podName:89ce251d-66b1-4990-8d8a-101dafeb009b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:09.328956012 +0000 UTC m=+4.033494495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs") pod "network-metrics-daemon-pwczh" (UID: "89ce251d-66b1-4990-8d8a-101dafeb009b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:08.329067 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:08.328915    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:08.329161 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:08.329077    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret podName:5271f263-1387-45f8-9d53-db6e1b9df066 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:09.329057014 +0000 UTC m=+4.033595503 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret") pod "global-pull-secret-syncer-fl64c" (UID: "5271f263-1387-45f8-9d53-db6e1b9df066") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:08.350541 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.350513    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:08.429987 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.429959    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9wkm\" (UniqueName: \"kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm\") pod \"network-check-target-qjv9w\" (UID: \"67bb572c-7609-4a7c-99ab-fcb047a0fc7b\") " pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:08.430148 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:08.430084    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:08.430148 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:08.430102    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:08.430148 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:08.430114    2576 projected.go:196] Error preparing data for projected volume kube-api-access-t9wkm for pod openshift-network-diagnostics/network-check-target-qjv9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:08.430326 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:08.430165    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm podName:67bb572c-7609-4a7c-99ab-fcb047a0fc7b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:09.430147465 +0000 UTC m=+4.134685946 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9wkm" (UniqueName: "kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm") pod "network-check-target-qjv9w" (UID: "67bb572c-7609-4a7c-99ab-fcb047a0fc7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:08.751618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.751588    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 07:44:07 +0000 UTC" deadline="2028-06-08 20:23:23.214071222 +0000 UTC"
Sep 14 07:49:08.751618 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.751614    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15204h34m14.462460751s"
Sep 14 07:49:08.795145 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:08.794985    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd19124b_82d8_4f39_a028_cfc81c6979c8.slice/crio-25b1cae94aa36d3c2528c7e287c13cff0a26985a6c5d2fa1e9c1250d0553edef WatchSource:0}: Error finding container 25b1cae94aa36d3c2528c7e287c13cff0a26985a6c5d2fa1e9c1250d0553edef: Status 404 returned error can't find the container with id 25b1cae94aa36d3c2528c7e287c13cff0a26985a6c5d2fa1e9c1250d0553edef
Sep 14 07:49:08.795593 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:08.795574    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a0e6d8_3d6e_4bf6_b50f_5f5d5f94ac41.slice/crio-b55274950b0fec6ac80b5cfedc64681ac6bf6bb7fa0362e6a4d29289410c1e3b WatchSource:0}: Error finding container b55274950b0fec6ac80b5cfedc64681ac6bf6bb7fa0362e6a4d29289410c1e3b: Status 404 returned error can't find the container with id b55274950b0fec6ac80b5cfedc64681ac6bf6bb7fa0362e6a4d29289410c1e3b
Sep 14 07:49:08.799541 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:08.799513    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08f5e8f4_1fcd_4186_a367_3697f02565ca.slice/crio-99dd8ffe377a2644f68ce9efa5043d203829896851f65d5c2139e46411d56d75 WatchSource:0}: Error finding container 99dd8ffe377a2644f68ce9efa5043d203829896851f65d5c2139e46411d56d75: Status 404 returned error can't find the container with id 99dd8ffe377a2644f68ce9efa5043d203829896851f65d5c2139e46411d56d75
Sep 14 07:49:08.803550 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:08.803526    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod505f1fe1_8e14_4b98_b9b8_9a90faf97b5d.slice/crio-1a9cfa9ce7f60b1b0b2632957bd504095193467da655d3446a3af55345be2857 WatchSource:0}: Error finding container 1a9cfa9ce7f60b1b0b2632957bd504095193467da655d3446a3af55345be2857: Status 404 returned error can't find the container with id 1a9cfa9ce7f60b1b0b2632957bd504095193467da655d3446a3af55345be2857
Sep 14 07:49:08.804986 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:08.804946    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1221aa_49ae_48fa_819a_f7d296b01557.slice/crio-89f62a65f940c1fa294d03d969ea1583d61c94a7c3d70f6a1ff99f4e7bea0683 WatchSource:0}: Error finding container 89f62a65f940c1fa294d03d969ea1583d61c94a7c3d70f6a1ff99f4e7bea0683: Status 404 returned error can't find the container with id 89f62a65f940c1fa294d03d969ea1583d61c94a7c3d70f6a1ff99f4e7bea0683
Sep 14 07:49:08.806121 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:08.806096    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc850c672_bf8a_4935_830c_db4cd90d3c6b.slice/crio-0117963b4e9ee17e49c7c77b07fe8015dee243b3b7b611947310a031ca88979c WatchSource:0}: Error finding container 0117963b4e9ee17e49c7c77b07fe8015dee243b3b7b611947310a031ca88979c: Status 404 returned error can't find the container with id 0117963b4e9ee17e49c7c77b07fe8015dee243b3b7b611947310a031ca88979c
Sep 14 07:49:08.807858 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:08.807831    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d441338_3dfd_4b6a_a535_f7af399bfe70.slice/crio-4cdd58bd62bf51d0fe349e32b307826bfb43dc9864e4ba69cd67dab08aef0980 WatchSource:0}: Error finding container 4cdd58bd62bf51d0fe349e32b307826bfb43dc9864e4ba69cd67dab08aef0980: Status 404 returned error can't find the container with id 4cdd58bd62bf51d0fe349e32b307826bfb43dc9864e4ba69cd67dab08aef0980
Sep 14 07:49:08.808227 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:08.808179    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a95c54_5096_456d_843c_62ee949876c2.slice/crio-b5b506aa811fa46a80fdd328610ff3d83da517579d0c0cc94814644bb3045ab0 WatchSource:0}: Error finding container b5b506aa811fa46a80fdd328610ff3d83da517579d0c0cc94814644bb3045ab0: Status 404 returned error can't find the container with id b5b506aa811fa46a80fdd328610ff3d83da517579d0c0cc94814644bb3045ab0
Sep 14 07:49:08.810003 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:08.809872    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe9c0768_7db5_401c_a564_6f06859a183d.slice/crio-6cd0b9999829d469bbcc432ca11bef40f7cdb320e9db282bc5c4fa278cf2d87e WatchSource:0}: Error finding container 6cd0b9999829d469bbcc432ca11bef40f7cdb320e9db282bc5c4fa278cf2d87e: Status 404 returned error can't find the container with id 6cd0b9999829d469bbcc432ca11bef40f7cdb320e9db282bc5c4fa278cf2d87e
Sep 14 07:49:08.832327 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.832286    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:08.832419 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.832305    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:08.832484 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:08.832426    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:08.832543 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:08.832519    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:08.840316 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.840290    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pwcsr" event={"ID":"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d","Type":"ContainerStarted","Data":"1a9cfa9ce7f60b1b0b2632957bd504095193467da655d3446a3af55345be2857"}
Sep 14 07:49:08.841525 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.841498    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6bfdg" event={"ID":"55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41","Type":"ContainerStarted","Data":"b55274950b0fec6ac80b5cfedc64681ac6bf6bb7fa0362e6a4d29289410c1e3b"}
Sep 14 07:49:08.843048 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.843023    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j9vrm" event={"ID":"be9c0768-7db5-401c-a564-6f06859a183d","Type":"ContainerStarted","Data":"6cd0b9999829d469bbcc432ca11bef40f7cdb320e9db282bc5c4fa278cf2d87e"}
Sep 14 07:49:08.844260 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.844236    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qgb6w" event={"ID":"41a95c54-5096-456d-843c-62ee949876c2","Type":"ContainerStarted","Data":"b5b506aa811fa46a80fdd328610ff3d83da517579d0c0cc94814644bb3045ab0"}
Sep 14 07:49:08.846555 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.846523    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm" event={"ID":"3d441338-3dfd-4b6a-a535-f7af399bfe70","Type":"ContainerStarted","Data":"4cdd58bd62bf51d0fe349e32b307826bfb43dc9864e4ba69cd67dab08aef0980"}
Sep 14 07:49:08.849387 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.849364    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zkjj" event={"ID":"c850c672-bf8a-4935-830c-db4cd90d3c6b","Type":"ContainerStarted","Data":"0117963b4e9ee17e49c7c77b07fe8015dee243b3b7b611947310a031ca88979c"}
Sep 14 07:49:08.850902 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.850851    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47" event={"ID":"9e1221aa-49ae-48fa-819a-f7d296b01557","Type":"ContainerStarted","Data":"89f62a65f940c1fa294d03d969ea1583d61c94a7c3d70f6a1ff99f4e7bea0683"}
Sep 14 07:49:08.852581 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.852557    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5xps5" event={"ID":"08f5e8f4-1fcd-4186-a367-3697f02565ca","Type":"ContainerStarted","Data":"99dd8ffe377a2644f68ce9efa5043d203829896851f65d5c2139e46411d56d75"}
Sep 14 07:49:08.853997 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:08.853975    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vqbl2" event={"ID":"bd19124b-82d8-4f39-a028-cfc81c6979c8","Type":"ContainerStarted","Data":"25b1cae94aa36d3c2528c7e287c13cff0a26985a6c5d2fa1e9c1250d0553edef"}
Sep 14 07:49:09.337199 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:09.336372    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:09.337199 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:09.336436    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs\") pod \"network-metrics-daemon-pwczh\" (UID: \"89ce251d-66b1-4990-8d8a-101dafeb009b\") " pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:09.337199 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:09.336600    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:09.337199 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:09.336668    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs podName:89ce251d-66b1-4990-8d8a-101dafeb009b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:11.336646227 +0000 UTC m=+6.041184716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs") pod "network-metrics-daemon-pwczh" (UID: "89ce251d-66b1-4990-8d8a-101dafeb009b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:09.337199 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:09.337058    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:09.337199 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:09.337112    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret podName:5271f263-1387-45f8-9d53-db6e1b9df066 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:11.337096183 +0000 UTC m=+6.041634671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret") pod "global-pull-secret-syncer-fl64c" (UID: "5271f263-1387-45f8-9d53-db6e1b9df066") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:09.437537 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:09.437499    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9wkm\" (UniqueName: \"kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm\") pod \"network-check-target-qjv9w\" (UID: \"67bb572c-7609-4a7c-99ab-fcb047a0fc7b\") " pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:09.437765 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:09.437678    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:09.437856 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:09.437780    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:09.437856 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:09.437795    2576 projected.go:196] Error preparing data for projected volume kube-api-access-t9wkm for pod openshift-network-diagnostics/network-check-target-qjv9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:09.437856 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:09.437855    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm podName:67bb572c-7609-4a7c-99ab-fcb047a0fc7b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:11.43783538 +0000 UTC m=+6.142373866 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9wkm" (UniqueName: "kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm") pod "network-check-target-qjv9w" (UID: "67bb572c-7609-4a7c-99ab-fcb047a0fc7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:09.751916 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:09.751812    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 07:44:07 +0000 UTC" deadline="2028-02-08 22:52:25.196065315 +0000 UTC"
Sep 14 07:49:09.751916 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:09.751873    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12303h3m15.444197417s"
Sep 14 07:49:09.833589 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:09.833042    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:09.833589 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:09.833170    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:09.871557 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:09.870870    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal" event={"ID":"8c553a2f2a397bf530bccc9a54adec3a","Type":"ContainerStarted","Data":"78959243e48076b0d703c2f596c585f9e62faba0e68bb81c764addc9fd0e70eb"}
Sep 14 07:49:09.878913 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:09.878452    2576 generic.go:356] "Generic (PLEG): container finished" podID="cca7b62a5f07be9c100b06a15efd970c" containerID="71d20a6d029279e5b1b1c4cbfd4a596022d848fecdc1b7823f98400c58cc3668" exitCode=0
Sep 14 07:49:09.878913 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:09.878499    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" event={"ID":"cca7b62a5f07be9c100b06a15efd970c","Type":"ContainerDied","Data":"71d20a6d029279e5b1b1c4cbfd4a596022d848fecdc1b7823f98400c58cc3668"}
Sep 14 07:49:09.910768 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:09.909890    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal" podStartSLOduration=2.909871233 podStartE2EDuration="2.909871233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:09.892429901 +0000 UTC m=+4.596968408" watchObservedRunningTime="2026-09-14 07:49:09.909871233 +0000 UTC m=+4.614409740"
Sep 14 07:49:10.833754 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:10.833027    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:10.833754 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:10.833157    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:10.833754 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:10.833561    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:10.833754 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:10.833661    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:10.891367 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:10.891331    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" event={"ID":"cca7b62a5f07be9c100b06a15efd970c","Type":"ContainerStarted","Data":"7bb80124ef1d913e8765dd454800308320d6e02531ab1ab9f1ac7286555b30fd"}
Sep 14 07:49:11.359522 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:11.358748    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs\") pod \"network-metrics-daemon-pwczh\" (UID: \"89ce251d-66b1-4990-8d8a-101dafeb009b\") " pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:11.359522 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:11.358871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:11.359522 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:11.358990    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:11.359522 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:11.359048    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret podName:5271f263-1387-45f8-9d53-db6e1b9df066 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:15.359029611 +0000 UTC m=+10.063568093 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret") pod "global-pull-secret-syncer-fl64c" (UID: "5271f263-1387-45f8-9d53-db6e1b9df066") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:11.359522 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:11.359428    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:11.359522 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:11.359475    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs podName:89ce251d-66b1-4990-8d8a-101dafeb009b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:15.359460065 +0000 UTC m=+10.063998548 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs") pod "network-metrics-daemon-pwczh" (UID: "89ce251d-66b1-4990-8d8a-101dafeb009b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:11.460456 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:11.459875    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9wkm\" (UniqueName: \"kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm\") pod \"network-check-target-qjv9w\" (UID: \"67bb572c-7609-4a7c-99ab-fcb047a0fc7b\") " pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:11.460456 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:11.460035    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:11.460456 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:11.460053    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:11.460456 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:11.460066    2576 projected.go:196] Error preparing data for projected volume kube-api-access-t9wkm for pod openshift-network-diagnostics/network-check-target-qjv9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:11.460456 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:11.460120    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm podName:67bb572c-7609-4a7c-99ab-fcb047a0fc7b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:15.460102205 +0000 UTC m=+10.164640685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9wkm" (UniqueName: "kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm") pod "network-check-target-qjv9w" (UID: "67bb572c-7609-4a7c-99ab-fcb047a0fc7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:11.843186 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:11.842444    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:11.843186 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:11.842601    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:12.833058 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:12.833027    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:12.833285 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:12.833166    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:12.833285 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:12.833241    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:12.833418 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:12.833353    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:13.832555 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:13.832524    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:13.833015 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:13.832651    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:14.832257 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:14.832220    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:14.832434 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:14.832231    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:14.832434 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:14.832347    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:14.832575 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:14.832431    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:15.391918 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:15.391877    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:15.392114 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:15.391940    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs\") pod \"network-metrics-daemon-pwczh\" (UID: \"89ce251d-66b1-4990-8d8a-101dafeb009b\") " pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:15.392114 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:15.392055    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:15.392114 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:15.392114    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs podName:89ce251d-66b1-4990-8d8a-101dafeb009b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:23.392094936 +0000 UTC m=+18.096633436 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs") pod "network-metrics-daemon-pwczh" (UID: "89ce251d-66b1-4990-8d8a-101dafeb009b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:15.392338 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:15.392302    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:15.392392 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:15.392348    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret podName:5271f263-1387-45f8-9d53-db6e1b9df066 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:23.392334836 +0000 UTC m=+18.096873339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret") pod "global-pull-secret-syncer-fl64c" (UID: "5271f263-1387-45f8-9d53-db6e1b9df066") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:15.493111 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:15.493063    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9wkm\" (UniqueName: \"kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm\") pod \"network-check-target-qjv9w\" (UID: \"67bb572c-7609-4a7c-99ab-fcb047a0fc7b\") " pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:15.493352 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:15.493256    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:15.493352 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:15.493282    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:15.493352 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:15.493296    2576 projected.go:196] Error preparing data for projected volume kube-api-access-t9wkm for pod openshift-network-diagnostics/network-check-target-qjv9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:15.493506 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:15.493366    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm podName:67bb572c-7609-4a7c-99ab-fcb047a0fc7b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:23.493347401 +0000 UTC m=+18.197885902 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9wkm" (UniqueName: "kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm") pod "network-check-target-qjv9w" (UID: "67bb572c-7609-4a7c-99ab-fcb047a0fc7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:15.834309 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:15.834224    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:15.834784 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:15.834343    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:16.833831 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:16.832946    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:16.833831 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:16.833130    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:16.833831 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:16.833633    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:16.833831 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:16.833783    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:17.832510 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:17.832475    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:17.832966 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:17.832606    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:18.832698 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:18.832666    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:18.833130 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:18.832666    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:18.833130 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:18.832796    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:18.833130 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:18.832845    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:19.832780 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:19.832747    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:19.833168 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:19.832873    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:20.832719 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:20.832684    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:20.832719 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:20.832712    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:20.833172 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:20.832825    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:20.833172 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:20.832944    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:21.832169 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:21.832127    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:21.832330 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:21.832274    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:22.833104 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:22.833074    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:22.833581 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:22.833074    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:22.833581 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:22.833224    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:22.833581 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:22.833290    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:23.455875 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:23.455627    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs\") pod \"network-metrics-daemon-pwczh\" (UID: \"89ce251d-66b1-4990-8d8a-101dafeb009b\") " pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:23.456130 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:23.455782    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:23.456130 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:23.456010    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs podName:89ce251d-66b1-4990-8d8a-101dafeb009b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:39.455988875 +0000 UTC m=+34.160527369 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs") pod "network-metrics-daemon-pwczh" (UID: "89ce251d-66b1-4990-8d8a-101dafeb009b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:23.456130 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:23.455940    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:23.456130 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:23.456025    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:23.456130 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:23.456097    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret podName:5271f263-1387-45f8-9d53-db6e1b9df066 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:39.45607982 +0000 UTC m=+34.160618301 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret") pod "global-pull-secret-syncer-fl64c" (UID: "5271f263-1387-45f8-9d53-db6e1b9df066") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:23.556958 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:23.556926    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9wkm\" (UniqueName: \"kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm\") pod \"network-check-target-qjv9w\" (UID: \"67bb572c-7609-4a7c-99ab-fcb047a0fc7b\") " pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:23.557101 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:23.557074    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:23.557101 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:23.557091    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:23.557101 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:23.557102    2576 projected.go:196] Error preparing data for projected volume kube-api-access-t9wkm for pod openshift-network-diagnostics/network-check-target-qjv9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:23.557258 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:23.557154    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm podName:67bb572c-7609-4a7c-99ab-fcb047a0fc7b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:39.55714164 +0000 UTC m=+34.261680133 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9wkm" (UniqueName: "kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm") pod "network-check-target-qjv9w" (UID: "67bb572c-7609-4a7c-99ab-fcb047a0fc7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:23.832178 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:23.832103    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:23.832334 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:23.832243    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:24.832269 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:24.832240    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:24.832269 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:24.832257    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:24.832751 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:24.832351    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:24.832751 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:24.832511    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:25.833612 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:25.833575    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:25.834043 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:25.833666    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:26.832656 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.832446    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:26.832777 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:26.832753    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:26.832835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.832776    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:26.832880 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:26.832857    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:26.928306 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.928246    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qgb6w" event={"ID":"41a95c54-5096-456d-843c-62ee949876c2","Type":"ContainerStarted","Data":"7499a9f33e01b805a3918504fc7e63190ad271688ef5a6087ac7dac6f0c49b28"}
Sep 14 07:49:26.929922 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.929901    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm" event={"ID":"3d441338-3dfd-4b6a-a535-f7af399bfe70","Type":"ContainerStarted","Data":"a2387eab2e6d40648126677a0ad18cf831936b089c0b50e86050980e7c123c8e"}
Sep 14 07:49:26.931748 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.931722    2576 generic.go:356] "Generic (PLEG): container finished" podID="c850c672-bf8a-4935-830c-db4cd90d3c6b" containerID="3d12ee9e17f9afd7d9cfa18c7ec9764adbfb55c2813e66994fddc236e85a5eb2" exitCode=0
Sep 14 07:49:26.931852 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.931793    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zkjj" event={"ID":"c850c672-bf8a-4935-830c-db4cd90d3c6b","Type":"ContainerDied","Data":"3d12ee9e17f9afd7d9cfa18c7ec9764adbfb55c2813e66994fddc236e85a5eb2"}
Sep 14 07:49:26.934831 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.934809    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47" event={"ID":"9e1221aa-49ae-48fa-819a-f7d296b01557","Type":"ContainerStarted","Data":"25cc652cc387a6f6153b3847b3b20b99e3a498014789e8d89810c947cbc8532c"}
Sep 14 07:49:26.934916 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.934888    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47" event={"ID":"9e1221aa-49ae-48fa-819a-f7d296b01557","Type":"ContainerStarted","Data":"a5fdffc32b49971e3e473f2559ccb753b2d317d382623b920c24eeae15dcfe37"}
Sep 14 07:49:26.934965 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.934939    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47" event={"ID":"9e1221aa-49ae-48fa-819a-f7d296b01557","Type":"ContainerStarted","Data":"bd79d1dd30f768a910bf8e63fac021059a1e3620db428a275f28a8be4378ac0d"}
Sep 14 07:49:26.935015 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.934981    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47" event={"ID":"9e1221aa-49ae-48fa-819a-f7d296b01557","Type":"ContainerStarted","Data":"2edd3ddbf221ef6e38af4760e4c0c78597c1daf9d98f14e58fd98ea6b64e49ec"}
Sep 14 07:49:26.936589 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.936550    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5xps5" event={"ID":"08f5e8f4-1fcd-4186-a367-3697f02565ca","Type":"ContainerStarted","Data":"8989bccb2246f256bfd8c3ca5ea1a225ddc3167be9cf5513817f06abcba401c3"}
Sep 14 07:49:26.938468 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.938440    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vqbl2" event={"ID":"bd19124b-82d8-4f39-a028-cfc81c6979c8","Type":"ContainerStarted","Data":"21806d83827d8897635fbfda4857a783d749c8d3e00dc2b16db9c7599b38adbe"}
Sep 14 07:49:26.940019 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.939997    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pwcsr" event={"ID":"505f1fe1-8e14-4b98-b9b8-9a90faf97b5d","Type":"ContainerStarted","Data":"5e95bf102fb1472a686190b81c2d2c113aa62e2db2ce3574d765f9bc91a59ac7"}
Sep 14 07:49:26.941397 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.941376    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6bfdg" event={"ID":"55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41","Type":"ContainerStarted","Data":"ced657aa210e51fab1d027876d5cdbcfa43745e31d7204cddf35acae3559b744"}
Sep 14 07:49:26.942398 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.942363    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qgb6w" podStartSLOduration=3.330291759 podStartE2EDuration="20.942350952s" totalImagesPullingTime="17.612059193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:26.941701252 +0000 UTC m=+21.646239753" watchObservedRunningTime="2026-09-14 07:49:26.942350952 +0000 UTC m=+21.646889454"
Sep 14 07:49:26.942473 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.942429    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" podStartSLOduration=19.94242446 podStartE2EDuration="19.94242446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:10.908893896 +0000 UTC m=+5.613432400" watchObservedRunningTime="2026-09-14 07:49:26.94242446 +0000 UTC m=+21.646962960"
Sep 14 07:49:26.984558 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.984510    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vqbl2" podStartSLOduration=3.358329504 podStartE2EDuration="20.984495625s" totalImagesPullingTime="17.626166121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:26.984298386 +0000 UTC m=+21.688836879" watchObservedRunningTime="2026-09-14 07:49:26.984495625 +0000 UTC m=+21.689034128"
Sep 14 07:49:26.984715 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:26.984689    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pwcsr" podStartSLOduration=3.121457465 podStartE2EDuration="20.984677974s" totalImagesPullingTime="17.863220509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:26.967429483 +0000 UTC m=+21.671967996" watchObservedRunningTime="2026-09-14 07:49:26.984677974 +0000 UTC m=+21.689216477"
Sep 14 07:49:27.020667 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:27.020626    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6bfdg" podStartSLOduration=7.207244994 podStartE2EDuration="21.020609992s" totalImagesPullingTime="13.813364998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:27.020553123 +0000 UTC m=+21.725091626" watchObservedRunningTime="2026-09-14 07:49:27.020609992 +0000 UTC m=+21.725148495"
Sep 14 07:49:27.044263 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:27.044228    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5xps5" podStartSLOduration=3.29461112 podStartE2EDuration="21.044202428s" totalImagesPullingTime="17.749591308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:27.043766464 +0000 UTC m=+21.748304967" watchObservedRunningTime="2026-09-14 07:49:27.044202428 +0000 UTC m=+21.748740931"
Sep 14 07:49:27.833000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:27.832976    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:27.833190 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:27.833078    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:27.945145 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:27.945112    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j9vrm" event={"ID":"be9c0768-7db5-401c-a564-6f06859a183d","Type":"ContainerStarted","Data":"d00882c830f339aa1b5c5460c816884799fd591de293f0ce880407c0a40bf3b7"}
Sep 14 07:49:27.948889 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:27.948858    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47" event={"ID":"9e1221aa-49ae-48fa-819a-f7d296b01557","Type":"ContainerStarted","Data":"e326c5969f471b99e30cfd1bd9db669df11dd17c071ed009511a0436041720fc"}
Sep 14 07:49:27.948994 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:27.948938    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47" event={"ID":"9e1221aa-49ae-48fa-819a-f7d296b01557","Type":"ContainerStarted","Data":"4574ed48fb520fd2c0e28cbd43144198b9d2eb48d22d35992336658176e9dc14"}
Sep 14 07:49:28.094123 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:28.093880    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 14 07:49:28.701124 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:28.701091    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6bfdg"
Sep 14 07:49:28.701695 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:28.701666    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6bfdg"
Sep 14 07:49:28.768838 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:28.768732    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-14T07:49:28.094073167Z","UUID":"f21d7416-5ed5-4bea-9c66-37a088c8322c","Handler":null,"Name":"","Endpoint":""}
Sep 14 07:49:28.770379 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:28.770361    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 14 07:49:28.770496 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:28.770409    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 14 07:49:28.832690 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:28.832667    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:28.832690 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:28.832685    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:28.832865 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:28.832768    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:28.832910 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:28.832876    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:28.953934 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:28.953834    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm" event={"ID":"3d441338-3dfd-4b6a-a535-f7af399bfe70","Type":"ContainerStarted","Data":"8fd021f048fb9224e5074b52ca2efd6c130f54f43470c3d8defbdc44225e2abf"}
Sep 14 07:49:28.970745 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:28.970692    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j9vrm" podStartSLOduration=5.252766539 podStartE2EDuration="22.97067623s" totalImagesPullingTime="17.717909691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:28.97056807 +0000 UTC m=+23.675106572" watchObservedRunningTime="2026-09-14 07:49:28.97067623 +0000 UTC m=+23.675214733"
Sep 14 07:49:29.832328 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:29.832294    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:29.832498 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:29.832407    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:29.960104 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:29.960076    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47" event={"ID":"9e1221aa-49ae-48fa-819a-f7d296b01557","Type":"ContainerStarted","Data":"4e16bb1828cce3e54c7a4eba8eda142328b2c0745735d20d0ac86baefd01ed5d"}
Sep 14 07:49:29.962405 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:29.962369    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm" event={"ID":"3d441338-3dfd-4b6a-a535-f7af399bfe70","Type":"ContainerStarted","Data":"d09806d5574a42a6c2e4e9f23abcb7052f2c857eac31294b0bf1336e09b9c162"}
Sep 14 07:49:29.962526 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:29.962386    2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 07:49:29.979935 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:29.979895    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zkhdm" podStartSLOduration=4.080843726 podStartE2EDuration="23.979883862s" totalImagesPullingTime="19.899040136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:29.979521305 +0000 UTC m=+24.684059807" watchObservedRunningTime="2026-09-14 07:49:29.979883862 +0000 UTC m=+24.684422364"
Sep 14 07:49:30.832388 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:30.832357    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:30.832388 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:30.832396    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:30.832619 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:30.832498    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:30.832676 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:30.832642    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:31.832771 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:31.832569    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:31.833383 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:31.832880    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:31.968086 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:31.968055    2576 generic.go:356] "Generic (PLEG): container finished" podID="c850c672-bf8a-4935-830c-db4cd90d3c6b" containerID="9e6aa6982fc0eb50d863f5e9d368d51522291849f2ea38013cc0a7d20d136ca2" exitCode=0
Sep 14 07:49:31.968371 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:31.968158    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zkjj" event={"ID":"c850c672-bf8a-4935-830c-db4cd90d3c6b","Type":"ContainerDied","Data":"9e6aa6982fc0eb50d863f5e9d368d51522291849f2ea38013cc0a7d20d136ca2"}
Sep 14 07:49:31.972562 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:31.972539    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47" event={"ID":"9e1221aa-49ae-48fa-819a-f7d296b01557","Type":"ContainerStarted","Data":"718a221b20df8d1d0d3fa10395b33a00e1ba1a94701c60006fd7c8035ea2dde2"}
Sep 14 07:49:31.972825 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:31.972805    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:31.972937 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:31.972876    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:31.972937 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:31.972930    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:31.988384 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:31.988362    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:31.988716 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:31.988692    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:49:32.018295 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:32.018244    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47" podStartSLOduration=8.272854036 podStartE2EDuration="26.018231237s" totalImagesPullingTime="17.745377201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:32.017965223 +0000 UTC m=+26.722503726" watchObservedRunningTime="2026-09-14 07:49:32.018231237 +0000 UTC m=+26.722769730"
Sep 14 07:49:32.832196 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:32.832172    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:32.832345 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:32.832287    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:32.832383 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:32.832343    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:32.832455 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:32.832438    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:33.228006 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:33.227808    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fl64c"]
Sep 14 07:49:33.228572 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:33.228151    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:33.228572 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:33.228278    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:33.230838 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:33.230807    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qjv9w"]
Sep 14 07:49:33.230992 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:33.230910    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:33.231052 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:33.231000    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:33.231704 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:33.231682    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pwczh"]
Sep 14 07:49:33.231815 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:33.231786    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:33.231933 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:33.231915    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:33.979096 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:33.979067    2576 generic.go:356] "Generic (PLEG): container finished" podID="c850c672-bf8a-4935-830c-db4cd90d3c6b" containerID="a60e0b808b58c5af5c93229933134d11b225724d082357951cdb7894fc899854" exitCode=0
Sep 14 07:49:33.979242 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:33.979149    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zkjj" event={"ID":"c850c672-bf8a-4935-830c-db4cd90d3c6b","Type":"ContainerDied","Data":"a60e0b808b58c5af5c93229933134d11b225724d082357951cdb7894fc899854"}
Sep 14 07:49:34.832653 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:34.832620    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:34.833035 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:34.832627    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:34.833035 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:34.832721    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:34.833035 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:34.832640    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:34.833035 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:34.832797    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:34.833035 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:34.832902    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:35.986292 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:35.986246    2576 generic.go:356] "Generic (PLEG): container finished" podID="c850c672-bf8a-4935-830c-db4cd90d3c6b" containerID="e4a62174d83e4263026493a62edfabe099c984d756a3da627cc0da402a0769c8" exitCode=0
Sep 14 07:49:35.986722 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:35.986320    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zkjj" event={"ID":"c850c672-bf8a-4935-830c-db4cd90d3c6b","Type":"ContainerDied","Data":"e4a62174d83e4263026493a62edfabe099c984d756a3da627cc0da402a0769c8"}
Sep 14 07:49:36.832562 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:36.832528    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:36.832728 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:36.832542    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:36.832728 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:36.832638    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:36.832728 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:36.832646    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:36.832728 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:36.832701    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:36.832992 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:36.832801    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:37.338454 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:37.338422    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6bfdg"
Sep 14 07:49:37.338876 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:37.338545    2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 07:49:37.339017 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:37.338992    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6bfdg"
Sep 14 07:49:38.832858 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:38.832825    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:38.833411 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:38.832942    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:38.833411 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:38.832958    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwczh" podUID="89ce251d-66b1-4990-8d8a-101dafeb009b"
Sep 14 07:49:38.833411 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:38.833020    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qjv9w" podUID="67bb572c-7609-4a7c-99ab-fcb047a0fc7b"
Sep 14 07:49:38.833411 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:38.833034    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:38.833411 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:38.833113    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fl64c" podUID="5271f263-1387-45f8-9d53-db6e1b9df066"
Sep 14 07:49:39.475593 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.475563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:39.475754 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.475611    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs\") pod \"network-metrics-daemon-pwczh\" (UID: \"89ce251d-66b1-4990-8d8a-101dafeb009b\") " pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:39.475754 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.475719    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:39.475754 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.475744    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:39.475886 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.475792    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret podName:5271f263-1387-45f8-9d53-db6e1b9df066 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:11.47577175 +0000 UTC m=+66.180310244 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret") pod "global-pull-secret-syncer-fl64c" (UID: "5271f263-1387-45f8-9d53-db6e1b9df066") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:39.475886 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.475813    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs podName:89ce251d-66b1-4990-8d8a-101dafeb009b nodeName:}" failed. No retries permitted until 2026-09-14 07:50:11.475802469 +0000 UTC m=+66.180340953 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs") pod "network-metrics-daemon-pwczh" (UID: "89ce251d-66b1-4990-8d8a-101dafeb009b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:39.555096 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.555069    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeReady"
Sep 14 07:49:39.555256 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.555187    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 14 07:49:39.575998 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.575966    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9wkm\" (UniqueName: \"kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm\") pod \"network-check-target-qjv9w\" (UID: \"67bb572c-7609-4a7c-99ab-fcb047a0fc7b\") " pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:39.576169 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.576150    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:39.576243 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.576174    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:39.576243 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.576184    2576 projected.go:196] Error preparing data for projected volume kube-api-access-t9wkm for pod openshift-network-diagnostics/network-check-target-qjv9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:39.576243 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.576242    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm podName:67bb572c-7609-4a7c-99ab-fcb047a0fc7b nodeName:}" failed. No retries permitted until 2026-09-14 07:50:11.576226136 +0000 UTC m=+66.280764617 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9wkm" (UniqueName: "kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm") pod "network-check-target-qjv9w" (UID: "67bb572c-7609-4a7c-99ab-fcb047a0fc7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:39.605425 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.605399    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n5qtb"]
Sep 14 07:49:39.632558 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.632534    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pp6kf"]
Sep 14 07:49:39.632689 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.632634    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n5qtb"
Sep 14 07:49:39.635138 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.635117    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 14 07:49:39.635279 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.635140    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 14 07:49:39.635279 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.635236    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q8nsp\""
Sep 14 07:49:39.635419 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.635277    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 14 07:49:39.646577 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.646558    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n5qtb"]
Sep 14 07:49:39.646672 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.646591    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pp6kf"]
Sep 14 07:49:39.646672 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.646643    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:39.649181 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.649152    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 14 07:49:39.649309 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.649155    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 14 07:49:39.649392 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.649156    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gsd8p\""
Sep 14 07:49:39.686362 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.686334    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"]
Sep 14 07:49:39.700575 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.700555    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-756cc58c7d-c8fsp"]
Sep 14 07:49:39.700784 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.700763    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:49:39.703472 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.703433    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 14 07:49:39.705032 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.704924    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 14 07:49:39.705032 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.704941    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 14 07:49:39.705032 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.704947    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-sp7z2\""
Sep 14 07:49:39.705032 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.704995    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 14 07:49:39.714505 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.714485    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"]
Sep 14 07:49:39.714599 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.714524    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-756cc58c7d-c8fsp"]
Sep 14 07:49:39.714662 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.714606    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:39.717419 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.717320    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 14 07:49:39.717419 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.717342    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 14 07:49:39.717419 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.717354    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 14 07:49:39.717419 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.717376    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 14 07:49:39.717667 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.717530    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-pq28c\""
Sep 14 07:49:39.717667 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.717541    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 14 07:49:39.717667 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.717620    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 14 07:49:39.777343 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.777313    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd2vt\" (UniqueName: \"kubernetes.io/projected/719989fb-5c65-4237-9352-addb9eaf5da0-kube-api-access-qd2vt\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:39.777454 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.777352    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:49:39.777454 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.777428    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/719989fb-5c65-4237-9352-addb9eaf5da0-config-volume\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:39.777551 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.777496    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:39.777551 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.777531    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e5088055-c0a7-475b-8808-129b29373f89-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:49:39.777644 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.777562    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d25763c-a961-4ea4-8a12-70f9fc32945d-cert\") pod \"ingress-canary-n5qtb\" (UID: \"6d25763c-a961-4ea4-8a12-70f9fc32945d\") " pod="openshift-ingress-canary/ingress-canary-n5qtb"
Sep 14 07:49:39.777644 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.777585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccb29\" (UniqueName: \"kubernetes.io/projected/6d25763c-a961-4ea4-8a12-70f9fc32945d-kube-api-access-ccb29\") pod \"ingress-canary-n5qtb\" (UID: \"6d25763c-a961-4ea4-8a12-70f9fc32945d\") " pod="openshift-ingress-canary/ingress-canary-n5qtb"
Sep 14 07:49:39.777644 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.777609    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/719989fb-5c65-4237-9352-addb9eaf5da0-tmp-dir\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:39.777772 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.777665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c7ms\" (UniqueName: \"kubernetes.io/projected/e5088055-c0a7-475b-8808-129b29373f89-kube-api-access-2c7ms\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:49:39.866717 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.866649    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:39.878713 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.878684    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:49:39.878811 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.878717    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-default-certificate\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:39.878851 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.878814    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 07:49:39.878851 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.878833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/719989fb-5c65-4237-9352-addb9eaf5da0-config-volume\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:39.878914 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.878863    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls podName:e5088055-c0a7-475b-8808-129b29373f89 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:40.378849265 +0000 UTC m=+35.083387745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-fwqng" (UID: "e5088055-c0a7-475b-8808-129b29373f89") : secret "cluster-monitoring-operator-tls" not found
Sep 14 07:49:39.878914 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.878895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:39.878992 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.878914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e5088055-c0a7-475b-8808-129b29373f89-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:49:39.878992 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.878941    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhftw\" (UniqueName: \"kubernetes.io/projected/b7577aa4-cff4-4194-ad9d-6b7e6165413f-kube-api-access-zhftw\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:39.878992 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.878970    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:39.878992 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.878983    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:39.878992 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.878989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d25763c-a961-4ea4-8a12-70f9fc32945d-cert\") pod \"ingress-canary-n5qtb\" (UID: \"6d25763c-a961-4ea4-8a12-70f9fc32945d\") " pod="openshift-ingress-canary/ingress-canary-n5qtb"
Sep 14 07:49:39.879251 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.879006    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccb29\" (UniqueName: \"kubernetes.io/projected/6d25763c-a961-4ea4-8a12-70f9fc32945d-kube-api-access-ccb29\") pod \"ingress-canary-n5qtb\" (UID: \"6d25763c-a961-4ea4-8a12-70f9fc32945d\") " pod="openshift-ingress-canary/ingress-canary-n5qtb"
Sep 14 07:49:39.879251 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.879027    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/719989fb-5c65-4237-9352-addb9eaf5da0-tmp-dir\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:39.879251 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.879040    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls podName:719989fb-5c65-4237-9352-addb9eaf5da0 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:40.379022448 +0000 UTC m=+35.083560936 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls") pod "dns-default-pp6kf" (UID: "719989fb-5c65-4237-9352-addb9eaf5da0") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:39.879251 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.879113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c7ms\" (UniqueName: \"kubernetes.io/projected/e5088055-c0a7-475b-8808-129b29373f89-kube-api-access-2c7ms\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:49:39.879251 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.879153    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd2vt\" (UniqueName: \"kubernetes.io/projected/719989fb-5c65-4237-9352-addb9eaf5da0-kube-api-access-qd2vt\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:39.879251 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.879170    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-stats-auth\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:39.879251 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.879184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:39.879547 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.879345    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/719989fb-5c65-4237-9352-addb9eaf5da0-tmp-dir\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:39.879547 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.879401    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/719989fb-5c65-4237-9352-addb9eaf5da0-config-volume\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:39.879632 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.879594    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e5088055-c0a7-475b-8808-129b29373f89-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:49:39.883289 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.883271    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d25763c-a961-4ea4-8a12-70f9fc32945d-cert\") pod \"ingress-canary-n5qtb\" (UID: \"6d25763c-a961-4ea4-8a12-70f9fc32945d\") " pod="openshift-ingress-canary/ingress-canary-n5qtb"
Sep 14 07:49:39.887902 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.887785    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd2vt\" (UniqueName: \"kubernetes.io/projected/719989fb-5c65-4237-9352-addb9eaf5da0-kube-api-access-qd2vt\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:39.888001 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.887885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccb29\" (UniqueName: \"kubernetes.io/projected/6d25763c-a961-4ea4-8a12-70f9fc32945d-kube-api-access-ccb29\") pod \"ingress-canary-n5qtb\" (UID: \"6d25763c-a961-4ea4-8a12-70f9fc32945d\") " pod="openshift-ingress-canary/ingress-canary-n5qtb"
Sep 14 07:49:39.899806 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.899788    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c7ms\" (UniqueName: \"kubernetes.io/projected/e5088055-c0a7-475b-8808-129b29373f89-kube-api-access-2c7ms\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:49:39.942723 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.942704    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n5qtb"
Sep 14 07:49:39.951379 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.951354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:39.980412 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.980391    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-stats-auth\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:39.980505 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.980419    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:39.980505 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.980445    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-default-certificate\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:39.980505 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.980495    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhftw\" (UniqueName: \"kubernetes.io/projected/b7577aa4-cff4-4194-ad9d-6b7e6165413f-kube-api-access-zhftw\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:39.980620 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.980525    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:39.980620 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.980539    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 07:49:39.980620 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.980600    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs podName:b7577aa4-cff4-4194-ad9d-6b7e6165413f nodeName:}" failed. No retries permitted until 2026-09-14 07:49:40.480581905 +0000 UTC m=+35.185120402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs") pod "router-default-756cc58c7d-c8fsp" (UID: "b7577aa4-cff4-4194-ad9d-6b7e6165413f") : secret "router-metrics-certs-default" not found
Sep 14 07:49:39.980717 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:39.980655    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle podName:b7577aa4-cff4-4194-ad9d-6b7e6165413f nodeName:}" failed. No retries permitted until 2026-09-14 07:49:40.48063983 +0000 UTC m=+35.185178329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle") pod "router-default-756cc58c7d-c8fsp" (UID: "b7577aa4-cff4-4194-ad9d-6b7e6165413f") : configmap references non-existent config key: service-ca.crt
Sep 14 07:49:39.982757 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.982738    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-default-certificate\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:39.982851 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.982800    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-stats-auth\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:39.988428 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:39.988410    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhftw\" (UniqueName: \"kubernetes.io/projected/b7577aa4-cff4-4194-ad9d-6b7e6165413f-kube-api-access-zhftw\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:40.236550 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:40.236519    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n5qtb"]
Sep 14 07:49:40.238845 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:40.238816    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d25763c_a961_4ea4_8a12_70f9fc32945d.slice/crio-dd829fe048b3d4a9eb68a3d5ef2d5bed4602655e9afa77618f893251fefdce1f WatchSource:0}: Error finding container dd829fe048b3d4a9eb68a3d5ef2d5bed4602655e9afa77618f893251fefdce1f: Status 404 returned error can't find the container with id dd829fe048b3d4a9eb68a3d5ef2d5bed4602655e9afa77618f893251fefdce1f
Sep 14 07:49:40.384460 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:40.384435    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:40.384550 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:40.384514    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:49:40.384609 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:40.384589    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:40.384643 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:40.384616    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 07:49:40.384679 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:40.384655    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls podName:719989fb-5c65-4237-9352-addb9eaf5da0 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:41.384639349 +0000 UTC m=+36.089177830 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls") pod "dns-default-pp6kf" (UID: "719989fb-5c65-4237-9352-addb9eaf5da0") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:40.384679 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:40.384669    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls podName:e5088055-c0a7-475b-8808-129b29373f89 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:41.384663175 +0000 UTC m=+36.089201654 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-fwqng" (UID: "e5088055-c0a7-475b-8808-129b29373f89") : secret "cluster-monitoring-operator-tls" not found
Sep 14 07:49:40.485441 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:40.485382    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:40.485536 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:40.485461    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:40.485536 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:40.485513    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 07:49:40.485611 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:40.485562    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle podName:b7577aa4-cff4-4194-ad9d-6b7e6165413f nodeName:}" failed. No retries permitted until 2026-09-14 07:49:41.48555052 +0000 UTC m=+36.190089000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle") pod "router-default-756cc58c7d-c8fsp" (UID: "b7577aa4-cff4-4194-ad9d-6b7e6165413f") : configmap references non-existent config key: service-ca.crt
Sep 14 07:49:40.485611 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:40.485579    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs podName:b7577aa4-cff4-4194-ad9d-6b7e6165413f nodeName:}" failed. No retries permitted until 2026-09-14 07:49:41.485572393 +0000 UTC m=+36.190110872 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs") pod "router-default-756cc58c7d-c8fsp" (UID: "b7577aa4-cff4-4194-ad9d-6b7e6165413f") : secret "router-metrics-certs-default" not found
Sep 14 07:49:40.832760 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:40.832699    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:49:40.832901 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:40.832699    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:49:40.832901 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:40.832699    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:49:40.835403 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:40.835380    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 07:49:40.835522 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:40.835422    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vfw6g\""
Sep 14 07:49:40.835885 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:40.835868    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 07:49:40.835975 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:40.835877    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wkwvc\""
Sep 14 07:49:40.836244 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:40.836091    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 07:49:40.836244 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:40.836175    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 07:49:41.000262 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:41.000229    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n5qtb" event={"ID":"6d25763c-a961-4ea4-8a12-70f9fc32945d","Type":"ContainerStarted","Data":"dd829fe048b3d4a9eb68a3d5ef2d5bed4602655e9afa77618f893251fefdce1f"}
Sep 14 07:49:41.003457 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:41.003435    2576 generic.go:356] "Generic (PLEG): container finished" podID="c850c672-bf8a-4935-830c-db4cd90d3c6b" containerID="b74914d123493406aee9501bb84948189ba8ec46eea4aac52f6cf124c2469b2a" exitCode=0
Sep 14 07:49:41.003541 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:41.003484    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zkjj" event={"ID":"c850c672-bf8a-4935-830c-db4cd90d3c6b","Type":"ContainerDied","Data":"b74914d123493406aee9501bb84948189ba8ec46eea4aac52f6cf124c2469b2a"}
Sep 14 07:49:41.183636 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:41.183603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:41.243932 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:41.243898    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:41.393952 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:41.393923    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:49:41.394094 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:41.393975    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:41.394094 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:41.394063    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 07:49:41.394194 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:41.394127    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls podName:e5088055-c0a7-475b-8808-129b29373f89 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:43.394111637 +0000 UTC m=+38.098650116 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-fwqng" (UID: "e5088055-c0a7-475b-8808-129b29373f89") : secret "cluster-monitoring-operator-tls" not found
Sep 14 07:49:41.394194 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:41.394072    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:41.394303 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:41.394223    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls podName:719989fb-5c65-4237-9352-addb9eaf5da0 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:43.394191837 +0000 UTC m=+38.098730323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls") pod "dns-default-pp6kf" (UID: "719989fb-5c65-4237-9352-addb9eaf5da0") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:41.431657 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:41.431602    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:41.495045 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:41.495021    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:41.495152 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:41.495069    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:41.495195 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:41.495160    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 07:49:41.495195 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:41.495165    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle podName:b7577aa4-cff4-4194-ad9d-6b7e6165413f nodeName:}" failed. No retries permitted until 2026-09-14 07:49:43.495142519 +0000 UTC m=+38.199681004 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle") pod "router-default-756cc58c7d-c8fsp" (UID: "b7577aa4-cff4-4194-ad9d-6b7e6165413f") : configmap references non-existent config key: service-ca.crt
Sep 14 07:49:41.495195 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:41.495196    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs podName:b7577aa4-cff4-4194-ad9d-6b7e6165413f nodeName:}" failed. No retries permitted until 2026-09-14 07:49:43.495185539 +0000 UTC m=+38.199724019 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs") pod "router-default-756cc58c7d-c8fsp" (UID: "b7577aa4-cff4-4194-ad9d-6b7e6165413f") : secret "router-metrics-certs-default" not found
Sep 14 07:49:41.532838 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:41.532808    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:41.595281 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:41.595249    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:42.011046 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:42.011014    2576 generic.go:356] "Generic (PLEG): container finished" podID="c850c672-bf8a-4935-830c-db4cd90d3c6b" containerID="3a4e6f04f7c16bf9c30bcf7e724d42ceee9d3760fe8fc90eb2c3601af6feb7ab" exitCode=0
Sep 14 07:49:42.011608 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:42.011059    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zkjj" event={"ID":"c850c672-bf8a-4935-830c-db4cd90d3c6b","Type":"ContainerDied","Data":"3a4e6f04f7c16bf9c30bcf7e724d42ceee9d3760fe8fc90eb2c3601af6feb7ab"}
Sep 14 07:49:42.405286 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:42.405101    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:42.505379 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:42.505343    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:43.017404 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:43.017368    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zkjj" event={"ID":"c850c672-bf8a-4935-830c-db4cd90d3c6b","Type":"ContainerStarted","Data":"9d57e813d41b90a25924e82c77c8fd4e9052977c962f3faa47926fee5f779c2d"}
Sep 14 07:49:43.019240 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:43.019200    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n5qtb" event={"ID":"6d25763c-a961-4ea4-8a12-70f9fc32945d","Type":"ContainerStarted","Data":"2109a6eda13c31b5a526132b6c099dc0f8f655fd9d9590032eedb5f2a6e39ee9"}
Sep 14 07:49:43.040004 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:43.039956    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7zkjj" podStartSLOduration=8.830448293 podStartE2EDuration="37.039943017s" totalImagesPullingTime="28.209494724s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:43.039084968 +0000 UTC m=+37.743623470" watchObservedRunningTime="2026-09-14 07:49:43.039943017 +0000 UTC m=+37.744481516"
Sep 14 07:49:43.053702 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:43.053628    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n5qtb" podStartSLOduration=2.07504023 podStartE2EDuration="4.053616785s" totalImagesPullingTime="1.978576555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:43.052558804 +0000 UTC m=+37.757097306" watchObservedRunningTime="2026-09-14 07:49:43.053616785 +0000 UTC m=+37.758155287"
Sep 14 07:49:43.141732 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:43.141704    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:43.197670 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:43.197650    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:43.249860 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:43.249839    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:43.410368 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:43.410335    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:43.410445 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:43.410411    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:49:43.410500 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:43.410485    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:43.410500 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:43.410495    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 07:49:43.410565 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:43.410544    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls podName:719989fb-5c65-4237-9352-addb9eaf5da0 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:47.410530652 +0000 UTC m=+42.115069138 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls") pod "dns-default-pp6kf" (UID: "719989fb-5c65-4237-9352-addb9eaf5da0") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:43.410565 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:43.410557    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls podName:e5088055-c0a7-475b-8808-129b29373f89 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:47.410551615 +0000 UTC m=+42.115090094 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-fwqng" (UID: "e5088055-c0a7-475b-8808-129b29373f89") : secret "cluster-monitoring-operator-tls" not found
Sep 14 07:49:43.510705 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:43.510679    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:43.510781 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:43.510766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:43.510820 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:43.510790    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 07:49:43.510859 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:43.510831    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs podName:b7577aa4-cff4-4194-ad9d-6b7e6165413f nodeName:}" failed. No retries permitted until 2026-09-14 07:49:47.510817384 +0000 UTC m=+42.215355867 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs") pod "router-default-756cc58c7d-c8fsp" (UID: "b7577aa4-cff4-4194-ad9d-6b7e6165413f") : secret "router-metrics-certs-default" not found
Sep 14 07:49:43.510897 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:43.510870    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle podName:b7577aa4-cff4-4194-ad9d-6b7e6165413f nodeName:}" failed. No retries permitted until 2026-09-14 07:49:47.510857344 +0000 UTC m=+42.215395824 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle") pod "router-default-756cc58c7d-c8fsp" (UID: "b7577aa4-cff4-4194-ad9d-6b7e6165413f") : configmap references non-existent config key: service-ca.crt
Sep 14 07:49:43.831759 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:43.831733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:43.884550 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:43.884523    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:44.184453 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.184425    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:44.284700 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.284671    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:44.566158 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.566098    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5"]
Sep 14 07:49:44.568116 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.568098    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5"
Sep 14 07:49:44.570675 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.570641    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 14 07:49:44.570786 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.570698    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:49:44.570786 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.570641    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 14 07:49:44.570892 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.570823    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 14 07:49:44.571326 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.571303    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-72vxr\""
Sep 14 07:49:44.577957 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.577933    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5"]
Sep 14 07:49:44.696658 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.696626    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fb9mc"]
Sep 14 07:49:44.698735 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.698713    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:44.701132 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.701102    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 14 07:49:44.701277 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.701106    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 14 07:49:44.701277 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.701110    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 14 07:49:44.701277 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.701115    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bqkjn\""
Sep 14 07:49:44.701486 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.701467    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 14 07:49:44.717529 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.717498    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fb9mc"]
Sep 14 07:49:44.718846 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.718821    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c4e8db51-9e25-4dff-be31-b15688ddf1be-serving-cert\") pod \"service-ca-operator-cf7c776b7-29mn5\" (UID: \"c4e8db51-9e25-4dff-be31-b15688ddf1be\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5"
Sep 14 07:49:44.718967 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.718929    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w292\" (UniqueName: \"kubernetes.io/projected/c4e8db51-9e25-4dff-be31-b15688ddf1be-kube-api-access-9w292\") pod \"service-ca-operator-cf7c776b7-29mn5\" (UID: \"c4e8db51-9e25-4dff-be31-b15688ddf1be\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5"
Sep 14 07:49:44.719017 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.718980    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c4e8db51-9e25-4dff-be31-b15688ddf1be-config\") pod \"service-ca-operator-cf7c776b7-29mn5\" (UID: \"c4e8db51-9e25-4dff-be31-b15688ddf1be\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5"
Sep 14 07:49:44.720775 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.720742    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:44.779386 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.779365    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:44.819638 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.819594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w292\" (UniqueName: \"kubernetes.io/projected/c4e8db51-9e25-4dff-be31-b15688ddf1be-kube-api-access-9w292\") pod \"service-ca-operator-cf7c776b7-29mn5\" (UID: \"c4e8db51-9e25-4dff-be31-b15688ddf1be\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5"
Sep 14 07:49:44.819638 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.819635    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c4e8db51-9e25-4dff-be31-b15688ddf1be-config\") pod \"service-ca-operator-cf7c776b7-29mn5\" (UID: \"c4e8db51-9e25-4dff-be31-b15688ddf1be\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5"
Sep 14 07:49:44.819747 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.819676    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/816becf2-a3af-484e-8a99-f99437f46b0c-crio-socket\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:44.819747 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.819695    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c4e8db51-9e25-4dff-be31-b15688ddf1be-serving-cert\") pod \"service-ca-operator-cf7c776b7-29mn5\" (UID: \"c4e8db51-9e25-4dff-be31-b15688ddf1be\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5"
Sep 14 07:49:44.819747 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.819711    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/816becf2-a3af-484e-8a99-f99437f46b0c-data-volume\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:44.819857 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.819755    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:44.819857 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.819784    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:44.819857 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.819810    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vfng\" (UniqueName: \"kubernetes.io/projected/816becf2-a3af-484e-8a99-f99437f46b0c-kube-api-access-9vfng\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:44.820286 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.820270    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c4e8db51-9e25-4dff-be31-b15688ddf1be-config\") pod \"service-ca-operator-cf7c776b7-29mn5\" (UID: \"c4e8db51-9e25-4dff-be31-b15688ddf1be\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5"
Sep 14 07:49:44.823159 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.823134    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c4e8db51-9e25-4dff-be31-b15688ddf1be-serving-cert\") pod \"service-ca-operator-cf7c776b7-29mn5\" (UID: \"c4e8db51-9e25-4dff-be31-b15688ddf1be\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5"
Sep 14 07:49:44.827116 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.827096    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w292\" (UniqueName: \"kubernetes.io/projected/c4e8db51-9e25-4dff-be31-b15688ddf1be-kube-api-access-9w292\") pod \"service-ca-operator-cf7c776b7-29mn5\" (UID: \"c4e8db51-9e25-4dff-be31-b15688ddf1be\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5"
Sep 14 07:49:44.878397 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.878367    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5"
Sep 14 07:49:44.921049 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.921020    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/816becf2-a3af-484e-8a99-f99437f46b0c-crio-socket\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:44.921204 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.921060    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/816becf2-a3af-484e-8a99-f99437f46b0c-data-volume\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:44.921204 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.921130    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:44.921204 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.921160    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:44.921204 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.921184    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vfng\" (UniqueName: \"kubernetes.io/projected/816becf2-a3af-484e-8a99-f99437f46b0c-kube-api-access-9vfng\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:44.921418 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:44.921251    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:44.921418 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:44.921280    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:44.921418 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:44.921314    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls podName:816becf2-a3af-484e-8a99-f99437f46b0c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:45.421293879 +0000 UTC m=+40.125832359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fb9mc" (UID: "816becf2-a3af-484e-8a99-f99437f46b0c") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:44.921418 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.921320    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/816becf2-a3af-484e-8a99-f99437f46b0c-crio-socket\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:44.921418 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:44.921343    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm podName:816becf2-a3af-484e-8a99-f99437f46b0c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:45.421326728 +0000 UTC m=+40.125865226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fb9mc" (UID: "816becf2-a3af-484e-8a99-f99437f46b0c") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:44.921636 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.921424    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/816becf2-a3af-484e-8a99-f99437f46b0c-data-volume\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:44.930924 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:44.930866    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vfng\" (UniqueName: \"kubernetes.io/projected/816becf2-a3af-484e-8a99-f99437f46b0c-kube-api-access-9vfng\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:45.147367 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:45.147342    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5"]
Sep 14 07:49:45.149780 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:45.149756    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4e8db51_9e25_4dff_be31_b15688ddf1be.slice/crio-83c3815665e7eb155ee5b95fb74061fe0025e2fc442d6b3d0378bfcb919ff5fd WatchSource:0}: Error finding container 83c3815665e7eb155ee5b95fb74061fe0025e2fc442d6b3d0378bfcb919ff5fd: Status 404 returned error can't find the container with id 83c3815665e7eb155ee5b95fb74061fe0025e2fc442d6b3d0378bfcb919ff5fd
Sep 14 07:49:45.425987 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:45.425930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:45.425987 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:45.425957    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:45.426326 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:45.426051    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:45.426326 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:45.426088    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:45.426326 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:45.426098    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm podName:816becf2-a3af-484e-8a99-f99437f46b0c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:46.426085466 +0000 UTC m=+41.130623947 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fb9mc" (UID: "816becf2-a3af-484e-8a99-f99437f46b0c") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:45.426326 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:45.426147    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls podName:816becf2-a3af-484e-8a99-f99437f46b0c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:46.426132203 +0000 UTC m=+41.130670684 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fb9mc" (UID: "816becf2-a3af-484e-8a99-f99437f46b0c") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:46.027856 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.027819    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5" event={"ID":"c4e8db51-9e25-4dff-be31-b15688ddf1be","Type":"ContainerStarted","Data":"83c3815665e7eb155ee5b95fb74061fe0025e2fc442d6b3d0378bfcb919ff5fd"}
Sep 14 07:49:46.089551 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.089513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:46.089698 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.089527    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-jtrdh"]
Sep 14 07:49:46.113079 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.113057    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-jtrdh"]
Sep 14 07:49:46.113177 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.113154    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-jtrdh"
Sep 14 07:49:46.115589 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.115565    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 14 07:49:46.115691 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.115605    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 14 07:49:46.116393 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.116378    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-hxmds\""
Sep 14 07:49:46.183020 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.182994    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:46.234614 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.234593    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl9kh\" (UniqueName: \"kubernetes.io/projected/bea0a7a6-e3aa-4c65-a748-a93ba8ea5989-kube-api-access-kl9kh\") pod \"migrator-8b4d697c5-jtrdh\" (UID: \"bea0a7a6-e3aa-4c65-a748-a93ba8ea5989\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-jtrdh"
Sep 14 07:49:46.287183 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.287115    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:46.335530 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.335505    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl9kh\" (UniqueName: \"kubernetes.io/projected/bea0a7a6-e3aa-4c65-a748-a93ba8ea5989-kube-api-access-kl9kh\") pod \"migrator-8b4d697c5-jtrdh\" (UID: \"bea0a7a6-e3aa-4c65-a748-a93ba8ea5989\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-jtrdh"
Sep 14 07:49:46.343583 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.343554    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl9kh\" (UniqueName: \"kubernetes.io/projected/bea0a7a6-e3aa-4c65-a748-a93ba8ea5989-kube-api-access-kl9kh\") pod \"migrator-8b4d697c5-jtrdh\" (UID: \"bea0a7a6-e3aa-4c65-a748-a93ba8ea5989\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-jtrdh"
Sep 14 07:49:46.421967 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.421940    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-jtrdh"
Sep 14 07:49:46.436377 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.436005    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:46.436377 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.436030    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:46.436377 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:46.436169    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:46.436377 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:46.436200    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:46.436377 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:46.436269    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls podName:816becf2-a3af-484e-8a99-f99437f46b0c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:48.436247744 +0000 UTC m=+43.140786441 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fb9mc" (UID: "816becf2-a3af-484e-8a99-f99437f46b0c") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:46.436377 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:46.436289    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm podName:816becf2-a3af-484e-8a99-f99437f46b0c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:48.436278374 +0000 UTC m=+43.140816868 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fb9mc" (UID: "816becf2-a3af-484e-8a99-f99437f46b0c") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:46.861345 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:46.861319    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vqbl2_bd19124b-82d8-4f39-a028-cfc81c6979c8/dns-node-resolver/0.log"
Sep 14 07:49:47.262163 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:47.262107    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-jtrdh"]
Sep 14 07:49:47.322675 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:49:47.322653    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbea0a7a6_e3aa_4c65_a748_a93ba8ea5989.slice/crio-44cda59652c5de6bffdc6d70819dc5c007bcce80cd49d45cbe944bdd8806d15c WatchSource:0}: Error finding container 44cda59652c5de6bffdc6d70819dc5c007bcce80cd49d45cbe944bdd8806d15c: Status 404 returned error can't find the container with id 44cda59652c5de6bffdc6d70819dc5c007bcce80cd49d45cbe944bdd8806d15c
Sep 14 07:49:47.445351 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:47.445323    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:49:47.445660 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:47.445393    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:47.445660 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:47.445456    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 07:49:47.445660 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:47.445516    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls podName:e5088055-c0a7-475b-8808-129b29373f89 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:55.445501304 +0000 UTC m=+50.150039784 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-fwqng" (UID: "e5088055-c0a7-475b-8808-129b29373f89") : secret "cluster-monitoring-operator-tls" not found
Sep 14 07:49:47.445660 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:47.445524    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:47.445660 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:47.445566    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls podName:719989fb-5c65-4237-9352-addb9eaf5da0 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:55.445554261 +0000 UTC m=+50.150092741 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls") pod "dns-default-pp6kf" (UID: "719989fb-5c65-4237-9352-addb9eaf5da0") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:47.546002 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:47.545926    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:47.546153 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:47.546037    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 07:49:47.546153 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:47.546094    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs podName:b7577aa4-cff4-4194-ad9d-6b7e6165413f nodeName:}" failed. No retries permitted until 2026-09-14 07:49:55.546079099 +0000 UTC m=+50.250617579 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs") pod "router-default-756cc58c7d-c8fsp" (UID: "b7577aa4-cff4-4194-ad9d-6b7e6165413f") : secret "router-metrics-certs-default" not found
Sep 14 07:49:47.546153 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:47.546142    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:47.546340 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:47.546259    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle podName:b7577aa4-cff4-4194-ad9d-6b7e6165413f nodeName:}" failed. No retries permitted until 2026-09-14 07:49:55.54625031 +0000 UTC m=+50.250788790 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle") pod "router-default-756cc58c7d-c8fsp" (UID: "b7577aa4-cff4-4194-ad9d-6b7e6165413f") : configmap references non-existent config key: service-ca.crt
Sep 14 07:49:47.661055 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:47.661034    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qgb6w_41a95c54-5096-456d-843c-62ee949876c2/node-ca/0.log"
Sep 14 07:49:48.033633 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:48.033571    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5" event={"ID":"c4e8db51-9e25-4dff-be31-b15688ddf1be","Type":"ContainerStarted","Data":"491b9df3d96541627fb8bc43764fdf0dffce05fff83bfb9df1091df2c13f46fb"}
Sep 14 07:49:48.034921 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:48.034888    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-jtrdh" event={"ID":"bea0a7a6-e3aa-4c65-a748-a93ba8ea5989","Type":"ContainerStarted","Data":"44cda59652c5de6bffdc6d70819dc5c007bcce80cd49d45cbe944bdd8806d15c"}
Sep 14 07:49:48.048798 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:48.048748    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5" podStartSLOduration=1.8424106820000001 podStartE2EDuration="4.048729878s" totalImagesPullingTime="2.206319196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:48.047769957 +0000 UTC m=+42.752308461" watchObservedRunningTime="2026-09-14 07:49:48.048729878 +0000 UTC m=+42.753268380"
Sep 14 07:49:48.451645 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:48.451614    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:48.451645 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:48.451649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:48.452162 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:48.451733    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:48.452162 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:48.451787    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:48.452162 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:48.451855    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls podName:816becf2-a3af-484e-8a99-f99437f46b0c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:52.451834047 +0000 UTC m=+47.156372529 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fb9mc" (UID: "816becf2-a3af-484e-8a99-f99437f46b0c") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:48.452162 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:48.451886    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm podName:816becf2-a3af-484e-8a99-f99437f46b0c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:52.451875611 +0000 UTC m=+47.156414092 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fb9mc" (UID: "816becf2-a3af-484e-8a99-f99437f46b0c") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:48.460738 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:48.460718    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n5qtb_6d25763c-a961-4ea4-8a12-70f9fc32945d/serve-healthcheck-canary/0.log"
Sep 14 07:49:48.651933 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:48.651910    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:48.707304 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:48.707235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:49.038460 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:49.038430    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-jtrdh" event={"ID":"bea0a7a6-e3aa-4c65-a748-a93ba8ea5989","Type":"ContainerStarted","Data":"b0ec61b63e043741b7a811d4c61f2404d2777395646aa0a9cbc9af5d489d2baf"}
Sep 14 07:49:49.089017 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:49.088993    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:49.356532 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:49.356505    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:49.458447 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:49.458420    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:50.042559 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:50.042526    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-jtrdh" event={"ID":"bea0a7a6-e3aa-4c65-a748-a93ba8ea5989","Type":"ContainerStarted","Data":"16b8cc87aeaca527fe5df8eb787556a82920130b406b01468e12a6e5e7d54826"}
Sep 14 07:49:50.058200 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:50.058162    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-jtrdh" podStartSLOduration=2.546523671 podStartE2EDuration="4.058149372s" totalImagesPullingTime="1.511625701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:50.057147363 +0000 UTC m=+44.761685865" watchObservedRunningTime="2026-09-14 07:49:50.058149372 +0000 UTC m=+44.762687919"
Sep 14 07:49:50.137425 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:50.137403    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:50.235592 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:50.235568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:51.002877 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:51.002830    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:51.059165 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:51.059139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:52.314561 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:52.314528    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:52.479974 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:52.479943    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:52.480129 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:52.479983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:49:52.480129 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:52.480076    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:52.480129 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:52.480093    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:52.480129 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:52.480157    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm podName:816becf2-a3af-484e-8a99-f99437f46b0c nodeName:}" failed. No retries permitted until 2026-09-14 07:50:00.48013062 +0000 UTC m=+55.184669114 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fb9mc" (UID: "816becf2-a3af-484e-8a99-f99437f46b0c") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:52.480419 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:52.480177    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls podName:816becf2-a3af-484e-8a99-f99437f46b0c nodeName:}" failed. No retries permitted until 2026-09-14 07:50:00.480167021 +0000 UTC m=+55.184705501 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fb9mc" (UID: "816becf2-a3af-484e-8a99-f99437f46b0c") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:54.828629 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:54.828597    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:54.888798 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:54.887290    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:55.472224 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:55.472177    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:55.501736 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:55.501710    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:49:55.501859 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:55.501768    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:49:55.501918 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:55.501872    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 07:49:55.501957 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:55.501935    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls podName:e5088055-c0a7-475b-8808-129b29373f89 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:11.501917752 +0000 UTC m=+66.206456240 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-fwqng" (UID: "e5088055-c0a7-475b-8808-129b29373f89") : secret "cluster-monitoring-operator-tls" not found
Sep 14 07:49:55.502014 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:55.501952    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:55.502014 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:55.502008    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls podName:719989fb-5c65-4237-9352-addb9eaf5da0 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:11.501993177 +0000 UTC m=+66.206531662 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls") pod "dns-default-pp6kf" (UID: "719989fb-5c65-4237-9352-addb9eaf5da0") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:55.582644 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:55.582619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:55.602825 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:55.602801    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:55.602919 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:55.602878    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:49:55.602975 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:55.602929    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 07:49:55.603014 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:55.602990    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs podName:b7577aa4-cff4-4194-ad9d-6b7e6165413f nodeName:}" failed. No retries permitted until 2026-09-14 07:50:11.602962557 +0000 UTC m=+66.307501043 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs") pod "router-default-756cc58c7d-c8fsp" (UID: "b7577aa4-cff4-4194-ad9d-6b7e6165413f") : secret "router-metrics-certs-default" not found
Sep 14 07:49:55.603065 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:49:55.603057    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle podName:b7577aa4-cff4-4194-ad9d-6b7e6165413f nodeName:}" failed. No retries permitted until 2026-09-14 07:50:11.603049076 +0000 UTC m=+66.307587557 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle") pod "router-default-756cc58c7d-c8fsp" (UID: "b7577aa4-cff4-4194-ad9d-6b7e6165413f") : configmap references non-existent config key: service-ca.crt
Sep 14 07:49:55.685540 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:55.685513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:55.785117 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:55.785052    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:55.952041 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:55.952017    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:56.138562 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:56.138538    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:56.953510 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:56.953480    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:57.344729 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:57.344643    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:57.773655 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:57.773622    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:57.958677 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:57.958650    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:58.181596 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:58.181569    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:58.337480 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:58.337455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:59.098522 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:59.098491    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:59.197790 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:59.197761    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:59.255947 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:59.255918    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:59.533170 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:59.533144    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:59.732078 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:59.732050    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:59.940683 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:59.940651    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:59.996370 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:49:59.996342    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:00.133129 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:00.133105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:00.539798 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:00.539769    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:50:00.539958 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:00.539898    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:50:00.540311 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:00.540290    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/816becf2-a3af-484e-8a99-f99437f46b0c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:50:00.542080 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:00.542063    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/816becf2-a3af-484e-8a99-f99437f46b0c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fb9mc\" (UID: \"816becf2-a3af-484e-8a99-f99437f46b0c\") " pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:50:00.608116 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:00.608075    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fb9mc"
Sep 14 07:50:00.880873 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:00.880844    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fb9mc"]
Sep 14 07:50:00.883706 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:50:00.883682    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod816becf2_a3af_484e_8a99_f99437f46b0c.slice/crio-74b599fe7b5b559dc4adc440f67f2e318ffcde262b27cf2eba62964f01c4f02b WatchSource:0}: Error finding container 74b599fe7b5b559dc4adc440f67f2e318ffcde262b27cf2eba62964f01c4f02b: Status 404 returned error can't find the container with id 74b599fe7b5b559dc4adc440f67f2e318ffcde262b27cf2eba62964f01c4f02b
Sep 14 07:50:00.960081 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:00.960057    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:01.068664 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:01.068637    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fb9mc" event={"ID":"816becf2-a3af-484e-8a99-f99437f46b0c","Type":"ContainerStarted","Data":"6c66787c58361d5144fcb6ecf56e5fc44146d7eef69788c2dc6f6951efa83fbb"}
Sep 14 07:50:01.068754 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:01.068688    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fb9mc" event={"ID":"816becf2-a3af-484e-8a99-f99437f46b0c","Type":"ContainerStarted","Data":"74b599fe7b5b559dc4adc440f67f2e318ffcde262b27cf2eba62964f01c4f02b"}
Sep 14 07:50:01.161300 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:01.161243    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:01.636842 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:01.636776    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:01.736401 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:01.736379    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:02.073708 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:02.073634    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fb9mc" event={"ID":"816becf2-a3af-484e-8a99-f99437f46b0c","Type":"ContainerStarted","Data":"beb37550bb6304dfa10f6b566a84c74a1bdcad7c2f03f879a5677723a3360e2b"}
Sep 14 07:50:02.133756 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:02.133728    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:02.338901 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:02.338842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:02.741053 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:02.741023    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:03.538037 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:03.538009    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:03.938860 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:03.938830    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:03.990596 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:03.990572    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6n47"
Sep 14 07:50:04.081728 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:04.081703    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fb9mc" event={"ID":"816becf2-a3af-484e-8a99-f99437f46b0c","Type":"ContainerStarted","Data":"2d00466bf221b7443e561b4e987cd01b39173fea6e85b61c428b2740f534e1d1"}
Sep 14 07:50:04.099906 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:04.099858    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fb9mc" podStartSLOduration=18.044955406 podStartE2EDuration="20.09983632s" totalImagesPullingTime="2.054880914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:04.099110641 +0000 UTC m=+58.803649143" watchObservedRunningTime="2026-09-14 07:50:04.09983632 +0000 UTC m=+58.804374821"
Sep 14 07:50:04.132750 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:04.132729    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:04.330801 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:04.330746    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:04.533781 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:04.533757    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:05.336874 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:05.336843    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:05.534350 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:05.534320    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:05.741343 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:05.741321    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:05.876224 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:05.876182    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:05.977341 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:05.977318    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:06.080721 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:06.080670    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:06.181074 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:06.181050    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:06.279375 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:06.279346    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:06.379009 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:06.378986    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:06.560948 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:06.560919    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:06.656932 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:06.656872    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:07.583672 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:07.583639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:07.636628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:07.636603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:07.764444 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:07.764423    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:07.966835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:07.966814    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:08.288741 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:08.288679    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:11.522590 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.522564    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:50:11.523087 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.522617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:50:11.523087 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.522667    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs\") pod \"network-metrics-daemon-pwczh\" (UID: \"89ce251d-66b1-4990-8d8a-101dafeb009b\") " pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:50:11.523087 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.522715    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:50:11.524866 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.524829    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 07:50:11.525029 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.525009    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/719989fb-5c65-4237-9352-addb9eaf5da0-metrics-tls\") pod \"dns-default-pp6kf\" (UID: \"719989fb-5c65-4237-9352-addb9eaf5da0\") " pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:50:11.525413 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.525395    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5088055-c0a7-475b-8808-129b29373f89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fwqng\" (UID: \"e5088055-c0a7-475b-8808-129b29373f89\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:50:11.525519 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.525505    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 07:50:11.535063 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.535041    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5271f263-1387-45f8-9d53-db6e1b9df066-original-pull-secret\") pod \"global-pull-secret-syncer-fl64c\" (UID: \"5271f263-1387-45f8-9d53-db6e1b9df066\") " pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:50:11.535159 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.535143    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ce251d-66b1-4990-8d8a-101dafeb009b-metrics-certs\") pod \"network-metrics-daemon-pwczh\" (UID: \"89ce251d-66b1-4990-8d8a-101dafeb009b\") " pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:50:11.623503 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.623479    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:50:11.623587 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.623512    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9wkm\" (UniqueName: \"kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm\") pod \"network-check-target-qjv9w\" (UID: \"67bb572c-7609-4a7c-99ab-fcb047a0fc7b\") " pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:50:11.623587 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.623539    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:50:11.624075 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.624056    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7577aa4-cff4-4194-ad9d-6b7e6165413f-service-ca-bundle\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:50:11.625359 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.625341    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7577aa4-cff4-4194-ad9d-6b7e6165413f-metrics-certs\") pod \"router-default-756cc58c7d-c8fsp\" (UID: \"b7577aa4-cff4-4194-ad9d-6b7e6165413f\") " pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:50:11.625847 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.625827    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 07:50:11.636450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.636427    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 07:50:11.647661 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.647642    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9wkm\" (UniqueName: \"kubernetes.io/projected/67bb572c-7609-4a7c-99ab-fcb047a0fc7b-kube-api-access-t9wkm\") pod \"network-check-target-qjv9w\" (UID: \"67bb572c-7609-4a7c-99ab-fcb047a0fc7b\") " pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:50:11.746621 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.746595    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vfw6g\""
Sep 14 07:50:11.750659 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.750645    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fl64c"
Sep 14 07:50:11.754778 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.754762    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwczh"
Sep 14 07:50:11.758325 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.758300    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gsd8p\""
Sep 14 07:50:11.758431 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.758415    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wkwvc\""
Sep 14 07:50:11.766904 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.766887    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:50:11.766993 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.766921    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:50:11.814719 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.814699    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-sp7z2\""
Sep 14 07:50:11.823472 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.823448    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"
Sep 14 07:50:11.825744 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.825720    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-pq28c\""
Sep 14 07:50:11.834928 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:11.834398    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:50:12.051374 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.051308    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fl64c"]
Sep 14 07:50:12.054261 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:50:12.054238    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5271f263_1387_45f8_9d53_db6e1b9df066.slice/crio-08e898d5076f7580e18d2e96e8cb853d9634d7104f36836876cd9c49fd3c3ac6 WatchSource:0}: Error finding container 08e898d5076f7580e18d2e96e8cb853d9634d7104f36836876cd9c49fd3c3ac6: Status 404 returned error can't find the container with id 08e898d5076f7580e18d2e96e8cb853d9634d7104f36836876cd9c49fd3c3ac6
Sep 14 07:50:12.086320 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.086298    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pwczh"]
Sep 14 07:50:12.088959 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:50:12.088931    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ce251d_66b1_4990_8d8a_101dafeb009b.slice/crio-1c666f763d85a31e355c40721929fde969a50bf4999c55ec3ea7e6569464edb6 WatchSource:0}: Error finding container 1c666f763d85a31e355c40721929fde969a50bf4999c55ec3ea7e6569464edb6: Status 404 returned error can't find the container with id 1c666f763d85a31e355c40721929fde969a50bf4999c55ec3ea7e6569464edb6
Sep 14 07:50:12.104669 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.104645    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qjv9w"]
Sep 14 07:50:12.106765 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.106737    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fl64c" event={"ID":"5271f263-1387-45f8-9d53-db6e1b9df066","Type":"ContainerStarted","Data":"08e898d5076f7580e18d2e96e8cb853d9634d7104f36836876cd9c49fd3c3ac6"}
Sep 14 07:50:12.107351 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:50:12.107329    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67bb572c_7609_4a7c_99ab_fcb047a0fc7b.slice/crio-bc282526799518aa07353ab78bbe15b946d06069b8882f9b64e8cc4dc7952ba3 WatchSource:0}: Error finding container bc282526799518aa07353ab78bbe15b946d06069b8882f9b64e8cc4dc7952ba3: Status 404 returned error can't find the container with id bc282526799518aa07353ab78bbe15b946d06069b8882f9b64e8cc4dc7952ba3
Sep 14 07:50:12.107968 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.107939    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pwczh" event={"ID":"89ce251d-66b1-4990-8d8a-101dafeb009b","Type":"ContainerStarted","Data":"1c666f763d85a31e355c40721929fde969a50bf4999c55ec3ea7e6569464edb6"}
Sep 14 07:50:12.128694 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.128642    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pp6kf"]
Sep 14 07:50:12.130294 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:50:12.130274    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod719989fb_5c65_4237_9352_addb9eaf5da0.slice/crio-c4723f5eb74f8f29d6c75e67fdf8cbe9126c65b210840c45e48dc34df1d3d762 WatchSource:0}: Error finding container c4723f5eb74f8f29d6c75e67fdf8cbe9126c65b210840c45e48dc34df1d3d762: Status 404 returned error can't find the container with id c4723f5eb74f8f29d6c75e67fdf8cbe9126c65b210840c45e48dc34df1d3d762
Sep 14 07:50:12.150831 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.150743    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng"]
Sep 14 07:50:12.152797 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:50:12.152780    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5088055_c0a7_475b_8808_129b29373f89.slice/crio-31284aeee53262a8f92e9a2f20fb16307c13e2a53648ac673713152082bfb4d5 WatchSource:0}: Error finding container 31284aeee53262a8f92e9a2f20fb16307c13e2a53648ac673713152082bfb4d5: Status 404 returned error can't find the container with id 31284aeee53262a8f92e9a2f20fb16307c13e2a53648ac673713152082bfb4d5
Sep 14 07:50:12.179500 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.179479    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-756cc58c7d-c8fsp"]
Sep 14 07:50:12.181836 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:50:12.181817    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7577aa4_cff4_4194_ad9d_6b7e6165413f.slice/crio-bb14d72a1c9aab9d32a949f3ee0d60138266268089548058453fd3940862f0c1 WatchSource:0}: Error finding container bb14d72a1c9aab9d32a949f3ee0d60138266268089548058453fd3940862f0c1: Status 404 returned error can't find the container with id bb14d72a1c9aab9d32a949f3ee0d60138266268089548058453fd3940862f0c1
Sep 14 07:50:12.455358 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.455331    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5565d489fd-qrhzm"]
Sep 14 07:50:12.468421 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.468400    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5565d489fd-qrhzm"
Sep 14 07:50:12.470677 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.470653    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5565d489fd-qrhzm"]
Sep 14 07:50:12.471056 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.471038    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-ddrjs\""
Sep 14 07:50:12.471142 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.471062    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 14 07:50:12.471403 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.471385    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 14 07:50:12.471946 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.471929    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 14 07:50:12.472041 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.471959    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 14 07:50:12.502041 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.502022    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:12.529640 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.529613    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16d973d7-77b0-460d-bab4-b3ec87ed360d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5565d489fd-qrhzm\" (UID: \"16d973d7-77b0-460d-bab4-b3ec87ed360d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5565d489fd-qrhzm"
Sep 14 07:50:12.529946 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.529668    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8q49\" (UniqueName: \"kubernetes.io/projected/16d973d7-77b0-460d-bab4-b3ec87ed360d-kube-api-access-w8q49\") pod \"managed-serviceaccount-addon-agent-5565d489fd-qrhzm\" (UID: \"16d973d7-77b0-460d-bab4-b3ec87ed360d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5565d489fd-qrhzm"
Sep 14 07:50:12.548971 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.548807    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-pgvq5"]
Sep 14 07:50:12.560930 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.560912    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"]
Sep 14 07:50:12.561083 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.561063    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pgvq5"
Sep 14 07:50:12.564568 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.564548    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pjjvx\""
Sep 14 07:50:12.564974 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.564935    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 14 07:50:12.565269 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.565185    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 14 07:50:12.573480 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.573444    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-pgvq5"]
Sep 14 07:50:12.573480 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.573481    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"]
Sep 14 07:50:12.573632 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.573552    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.575687 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.575668    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 14 07:50:12.576938 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.576916    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 14 07:50:12.577021 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.576943    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 14 07:50:12.577075 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.577056    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vfxc9\""
Sep 14 07:50:12.582980 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.582960    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 14 07:50:12.609664 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.609642    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:12.630031 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.630002    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-pgvq5\" (UID: \"8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pgvq5"
Sep 14 07:50:12.630137 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.630048    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-pgvq5\" (UID: \"8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pgvq5"
Sep 14 07:50:12.630137 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.630087    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8q49\" (UniqueName: \"kubernetes.io/projected/16d973d7-77b0-460d-bab4-b3ec87ed360d-kube-api-access-w8q49\") pod \"managed-serviceaccount-addon-agent-5565d489fd-qrhzm\" (UID: \"16d973d7-77b0-460d-bab4-b3ec87ed360d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5565d489fd-qrhzm"
Sep 14 07:50:12.630137 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.630112    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2355229a-ae08-4fb2-95bb-607e70de9907-ca-trust-extracted\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.630317 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.630141    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2355229a-ae08-4fb2-95bb-607e70de9907-registry-tls\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.630317 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.630195    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvnm9\" (UniqueName: \"kubernetes.io/projected/2355229a-ae08-4fb2-95bb-607e70de9907-kube-api-access-rvnm9\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.630317 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.630242    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2355229a-ae08-4fb2-95bb-607e70de9907-registry-certificates\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.630317 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.630260    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2355229a-ae08-4fb2-95bb-607e70de9907-installation-pull-secrets\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.630317 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.630277    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2355229a-ae08-4fb2-95bb-607e70de9907-bound-sa-token\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.630317 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.630302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16d973d7-77b0-460d-bab4-b3ec87ed360d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5565d489fd-qrhzm\" (UID: \"16d973d7-77b0-460d-bab4-b3ec87ed360d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5565d489fd-qrhzm"
Sep 14 07:50:12.630317 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.630317    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2355229a-ae08-4fb2-95bb-607e70de9907-trusted-ca\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.630643 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.630348    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2355229a-ae08-4fb2-95bb-607e70de9907-image-registry-private-configuration\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.633641 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.633619    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16d973d7-77b0-460d-bab4-b3ec87ed360d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5565d489fd-qrhzm\" (UID: \"16d973d7-77b0-460d-bab4-b3ec87ed360d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5565d489fd-qrhzm"
Sep 14 07:50:12.645798 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.645750    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8q49\" (UniqueName: \"kubernetes.io/projected/16d973d7-77b0-460d-bab4-b3ec87ed360d-kube-api-access-w8q49\") pod \"managed-serviceaccount-addon-agent-5565d489fd-qrhzm\" (UID: \"16d973d7-77b0-460d-bab4-b3ec87ed360d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5565d489fd-qrhzm"
Sep 14 07:50:12.702551 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.702521    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:12.731030 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.730970    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2355229a-ae08-4fb2-95bb-607e70de9907-trusted-ca\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.731030 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.731015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2355229a-ae08-4fb2-95bb-607e70de9907-image-registry-private-configuration\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.731188 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.731072    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-pgvq5\" (UID: \"8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pgvq5"
Sep 14 07:50:12.731188 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.731099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-pgvq5\" (UID: \"8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pgvq5"
Sep 14 07:50:12.731188 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.731134    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2355229a-ae08-4fb2-95bb-607e70de9907-ca-trust-extracted\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.731188 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.731163    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2355229a-ae08-4fb2-95bb-607e70de9907-registry-tls\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.731344 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.731188    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvnm9\" (UniqueName: \"kubernetes.io/projected/2355229a-ae08-4fb2-95bb-607e70de9907-kube-api-access-rvnm9\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.731344 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.731285    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2355229a-ae08-4fb2-95bb-607e70de9907-registry-certificates\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.731344 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.731311    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2355229a-ae08-4fb2-95bb-607e70de9907-installation-pull-secrets\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.731430 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.731346    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2355229a-ae08-4fb2-95bb-607e70de9907-bound-sa-token\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.733389 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.733368    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2355229a-ae08-4fb2-95bb-607e70de9907-registry-certificates\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.734569 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.734530    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2355229a-ae08-4fb2-95bb-607e70de9907-ca-trust-extracted\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.734665 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.734613    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2355229a-ae08-4fb2-95bb-607e70de9907-trusted-ca\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.735052 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.735033    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-pgvq5\" (UID: \"8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pgvq5"
Sep 14 07:50:12.742240 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.742173    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2355229a-ae08-4fb2-95bb-607e70de9907-installation-pull-secrets\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.742240 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.742174    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2355229a-ae08-4fb2-95bb-607e70de9907-image-registry-private-configuration\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.743036 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.743013    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-pgvq5\" (UID: \"8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pgvq5"
Sep 14 07:50:12.746885 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.746845    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvnm9\" (UniqueName: \"kubernetes.io/projected/2355229a-ae08-4fb2-95bb-607e70de9907-kube-api-access-rvnm9\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.747514 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.747483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2355229a-ae08-4fb2-95bb-607e70de9907-bound-sa-token\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.747908 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.747885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2355229a-ae08-4fb2-95bb-607e70de9907-registry-tls\") pod \"image-registry-7bc5c4ffc4-p2pqp\" (UID: \"2355229a-ae08-4fb2-95bb-607e70de9907\") " pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.791993 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.791970    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5565d489fd-qrhzm"
Sep 14 07:50:12.807687 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.807662    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:12.875887 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.875471    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pgvq5"
Sep 14 07:50:12.884156 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.884062    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:12.906770 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:12.906738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:13.092122 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:13.092055    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5565d489fd-qrhzm"]
Sep 14 07:50:13.097488 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:50:13.097460    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16d973d7_77b0_460d_bab4_b3ec87ed360d.slice/crio-7651215f4b9b5f4a56401edca261146f1d054af58c31a854494f6ea576371446 WatchSource:0}: Error finding container 7651215f4b9b5f4a56401edca261146f1d054af58c31a854494f6ea576371446: Status 404 returned error can't find the container with id 7651215f4b9b5f4a56401edca261146f1d054af58c31a854494f6ea576371446
Sep 14 07:50:13.116425 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:13.116377    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-756cc58c7d-c8fsp" event={"ID":"b7577aa4-cff4-4194-ad9d-6b7e6165413f","Type":"ContainerStarted","Data":"51a3e66566100d4ab8ff030e0323c4501f46488545f87a69bd8d0cd497c57f9e"}
Sep 14 07:50:13.116527 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:13.116430    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-756cc58c7d-c8fsp" event={"ID":"b7577aa4-cff4-4194-ad9d-6b7e6165413f","Type":"ContainerStarted","Data":"bb14d72a1c9aab9d32a949f3ee0d60138266268089548058453fd3940862f0c1"}
Sep 14 07:50:13.120201 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:13.120166    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng" event={"ID":"e5088055-c0a7-475b-8808-129b29373f89","Type":"ContainerStarted","Data":"31284aeee53262a8f92e9a2f20fb16307c13e2a53648ac673713152082bfb4d5"}
Sep 14 07:50:13.122088 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:13.122036    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qjv9w" event={"ID":"67bb572c-7609-4a7c-99ab-fcb047a0fc7b","Type":"ContainerStarted","Data":"bc282526799518aa07353ab78bbe15b946d06069b8882f9b64e8cc4dc7952ba3"}
Sep 14 07:50:13.123649 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:13.123625    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5565d489fd-qrhzm" event={"ID":"16d973d7-77b0-460d-bab4-b3ec87ed360d","Type":"ContainerStarted","Data":"7651215f4b9b5f4a56401edca261146f1d054af58c31a854494f6ea576371446"}
Sep 14 07:50:13.125293 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:13.125242    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pp6kf" event={"ID":"719989fb-5c65-4237-9352-addb9eaf5da0","Type":"ContainerStarted","Data":"c4723f5eb74f8f29d6c75e67fdf8cbe9126c65b210840c45e48dc34df1d3d762"}
Sep 14 07:50:13.135462 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:13.134536    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-756cc58c7d-c8fsp" podStartSLOduration=34.134519067 podStartE2EDuration="34.134519067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:13.133250104 +0000 UTC m=+67.837788606" watchObservedRunningTime="2026-09-14 07:50:13.134519067 +0000 UTC m=+67.839057570"
Sep 14 07:50:13.212446 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:13.212356    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-pgvq5"]
Sep 14 07:50:13.216236 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:50:13.216188    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c3cbd65_3720_44ff_a4c5_d8b6e7a65dde.slice/crio-ba79e1ced3ee5dc163390c8cee3fd72d1ee73c86cbc1ff679e425d6ecf15234b WatchSource:0}: Error finding container ba79e1ced3ee5dc163390c8cee3fd72d1ee73c86cbc1ff679e425d6ecf15234b: Status 404 returned error can't find the container with id ba79e1ced3ee5dc163390c8cee3fd72d1ee73c86cbc1ff679e425d6ecf15234b
Sep 14 07:50:13.262462 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:13.262399    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"]
Sep 14 07:50:13.760623 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:50:13.760584    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2355229a_ae08_4fb2_95bb_607e70de9907.slice/crio-fef1cfcc7d4c389ba0d25d339ab7548cc1cf7c859ffa8ad30c1393e10a336f8b WatchSource:0}: Error finding container fef1cfcc7d4c389ba0d25d339ab7548cc1cf7c859ffa8ad30c1393e10a336f8b: Status 404 returned error can't find the container with id fef1cfcc7d4c389ba0d25d339ab7548cc1cf7c859ffa8ad30c1393e10a336f8b
Sep 14 07:50:13.837786 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:13.837758    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:50:13.837906 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:13.837873    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:50:14.133188 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:14.133128    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp" event={"ID":"2355229a-ae08-4fb2-95bb-607e70de9907","Type":"ContainerStarted","Data":"c20d8491a824a23b420bc94b63194c5c5f586da4075f9ada919763ecb36d030d"}
Sep 14 07:50:14.133378 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:14.133271    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp" event={"ID":"2355229a-ae08-4fb2-95bb-607e70de9907","Type":"ContainerStarted","Data":"fef1cfcc7d4c389ba0d25d339ab7548cc1cf7c859ffa8ad30c1393e10a336f8b"}
Sep 14 07:50:14.134117 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:14.134093    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:14.136756 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:14.136716    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pgvq5" event={"ID":"8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde","Type":"ContainerStarted","Data":"ba79e1ced3ee5dc163390c8cee3fd72d1ee73c86cbc1ff679e425d6ecf15234b"}
Sep 14 07:50:14.136879 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:14.136771    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:50:14.139642 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:14.138668    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-756cc58c7d-c8fsp"
Sep 14 07:50:14.156448 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:14.156371    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp" podStartSLOduration=2.156353109 podStartE2EDuration="2.156353109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:14.154971202 +0000 UTC m=+68.859509732" watchObservedRunningTime="2026-09-14 07:50:14.156353109 +0000 UTC m=+68.860891613"
Sep 14 07:50:14.229888 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:14.229854    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:14.323140 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:14.323103    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:15.049563 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:15.049531    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:15.420855 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:15.420825    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:15.522201 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:15.522152    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:19.205674 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:19.205636    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:19.256463 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:19.256374    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:19.695695 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:19.695659    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:19.797123 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:19.797056    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:19.878390 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:19.878358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:19.962690 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:19.962663    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:20.159745 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.159711    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pwczh" event={"ID":"89ce251d-66b1-4990-8d8a-101dafeb009b","Type":"ContainerStarted","Data":"ed7c1d9647f2c0544fe77243a50c14526b507b030218abc3d7aca78b3ecfacec"}
Sep 14 07:50:20.159872 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.159766    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pwczh" event={"ID":"89ce251d-66b1-4990-8d8a-101dafeb009b","Type":"ContainerStarted","Data":"8470c6ae69e1baf259587d3ea01491569b903a2f9494666fdb6953278bf02a6f"}
Sep 14 07:50:20.161158 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.161132    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5565d489fd-qrhzm" event={"ID":"16d973d7-77b0-460d-bab4-b3ec87ed360d","Type":"ContainerStarted","Data":"8a74c0695cbeb3483a94f0fd82515b329a77322792d057a09bfd9154f39429f9"}
Sep 14 07:50:20.162897 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.162872    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pp6kf" event={"ID":"719989fb-5c65-4237-9352-addb9eaf5da0","Type":"ContainerStarted","Data":"8b1a1ed2d2588265ac0dffd3b68f0ae11f07f4c39985d17c6306c7a10d987dab"}
Sep 14 07:50:20.162995 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.162916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pp6kf" event={"ID":"719989fb-5c65-4237-9352-addb9eaf5da0","Type":"ContainerStarted","Data":"71d8e593b2c008d3d8c6dd823e28e58608c337f4a7e706ccaf5c574763392e57"}
Sep 14 07:50:20.162995 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.162947    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:50:20.164523 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.164500    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pgvq5" event={"ID":"8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde","Type":"ContainerStarted","Data":"6d79e2bf95dbd4e8fa47eb8004faa7f98789adff440dbeb0fa27344e3b33aaa4"}
Sep 14 07:50:20.165834 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.165816    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fl64c" event={"ID":"5271f263-1387-45f8-9d53-db6e1b9df066","Type":"ContainerStarted","Data":"7e57dde7c9217ba27f6f9429fd417972fd1a3d349516c23ab4643abab0cd02fc"}
Sep 14 07:50:20.167187 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.167166    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng" event={"ID":"e5088055-c0a7-475b-8808-129b29373f89","Type":"ContainerStarted","Data":"cb4bc8adcebe4fc51aab8482b713d5dc158dede738651f87aa17d4e072407e84"}
Sep 14 07:50:20.168665 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.168647    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qjv9w" event={"ID":"67bb572c-7609-4a7c-99ab-fcb047a0fc7b","Type":"ContainerStarted","Data":"0a7ba001d5daa2b5a31c58c9decc8901a80a790281cca1ed51b995b4445a74d8"}
Sep 14 07:50:20.168748 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.168696    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:50:20.177651 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.177609    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pwczh" podStartSLOduration=67.61488437 podStartE2EDuration="1m14.177597833s" totalImagesPullingTime="6.562713463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:20.176769735 +0000 UTC m=+74.881308237" watchObservedRunningTime="2026-09-14 07:50:20.177597833 +0000 UTC m=+74.882136334"
Sep 14 07:50:20.190269 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.190232    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qjv9w" podStartSLOduration=67.009274682 podStartE2EDuration="1m14.190198959s" totalImagesPullingTime="7.180924277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:20.189853431 +0000 UTC m=+74.894391936" watchObservedRunningTime="2026-09-14 07:50:20.190198959 +0000 UTC m=+74.894737460"
Sep 14 07:50:20.205464 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.205423    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5565d489fd-qrhzm" podStartSLOduration=2.005055697 podStartE2EDuration="8.205411691s" totalImagesPullingTime="6.200355994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:20.204820687 +0000 UTC m=+74.909359189" watchObservedRunningTime="2026-09-14 07:50:20.205411691 +0000 UTC m=+74.909950227"
Sep 14 07:50:20.221485 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.221445    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pp6kf" podStartSLOduration=34.096452915 podStartE2EDuration="41.221432978s" totalImagesPullingTime="7.124980063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:20.220652238 +0000 UTC m=+74.925190740" watchObservedRunningTime="2026-09-14 07:50:20.221432978 +0000 UTC m=+74.925971479"
Sep 14 07:50:20.235120 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.235081    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fwqng" podStartSLOduration=34.099187102 podStartE2EDuration="41.235071783s" totalImagesPullingTime="7.135884681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:20.234060517 +0000 UTC m=+74.938599019" watchObservedRunningTime="2026-09-14 07:50:20.235071783 +0000 UTC m=+74.939610284"
Sep 14 07:50:20.248904 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.248861    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pgvq5" podStartSLOduration=2.174618434 podStartE2EDuration="8.248850138s" totalImagesPullingTime="6.074231704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:20.248247846 +0000 UTC m=+74.952786350" watchObservedRunningTime="2026-09-14 07:50:20.248850138 +0000 UTC m=+74.953388640"
Sep 14 07:50:20.263646 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.263613    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fl64c" podStartSLOduration=67.029460556 podStartE2EDuration="1m14.263601895s" totalImagesPullingTime="7.234141339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:20.262946348 +0000 UTC m=+74.967484853" watchObservedRunningTime="2026-09-14 07:50:20.263601895 +0000 UTC m=+74.968140397"
Sep 14 07:50:20.285573 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.285550    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:20.343709 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.343688    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:20.406682 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.406657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:20.642674 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.642646    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:20.699156 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:20.699134    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:23.986965 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:23.986924    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:24.034876 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:24.034843    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:24.619790 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:24.619743    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:24.672947 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:24.672918    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:24.885765 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:24.885701    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:24.921926 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:24.921860    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"]
Sep 14 07:50:24.925146 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:24.925125    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:24.927706 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:24.927686    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 14 07:50:24.927820 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:24.927723    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 14 07:50:24.928696 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:24.928674    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-8pths\""
Sep 14 07:50:24.928812 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:24.928712    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 14 07:50:24.934739 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:24.934719    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"]
Sep 14 07:50:24.947806 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:24.947780    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:25.022744 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.022721    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/725b3545-1cdb-48bf-8a09-d20468e44525-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-b4bg7\" (UID: \"725b3545-1cdb-48bf-8a09-d20468e44525\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:25.023038 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.022753    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nvpq\" (UniqueName: \"kubernetes.io/projected/725b3545-1cdb-48bf-8a09-d20468e44525-kube-api-access-8nvpq\") pod \"prometheus-operator-68bcd7f987-b4bg7\" (UID: \"725b3545-1cdb-48bf-8a09-d20468e44525\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:25.023038 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.022779    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/725b3545-1cdb-48bf-8a09-d20468e44525-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-b4bg7\" (UID: \"725b3545-1cdb-48bf-8a09-d20468e44525\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:25.023038 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.022804    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/725b3545-1cdb-48bf-8a09-d20468e44525-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-b4bg7\" (UID: \"725b3545-1cdb-48bf-8a09-d20468e44525\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:25.123848 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.123811    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nvpq\" (UniqueName: \"kubernetes.io/projected/725b3545-1cdb-48bf-8a09-d20468e44525-kube-api-access-8nvpq\") pod \"prometheus-operator-68bcd7f987-b4bg7\" (UID: \"725b3545-1cdb-48bf-8a09-d20468e44525\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:25.123957 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.123863    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/725b3545-1cdb-48bf-8a09-d20468e44525-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-b4bg7\" (UID: \"725b3545-1cdb-48bf-8a09-d20468e44525\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:25.123957 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.123895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/725b3545-1cdb-48bf-8a09-d20468e44525-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-b4bg7\" (UID: \"725b3545-1cdb-48bf-8a09-d20468e44525\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:25.124072 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.124053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/725b3545-1cdb-48bf-8a09-d20468e44525-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-b4bg7\" (UID: \"725b3545-1cdb-48bf-8a09-d20468e44525\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:25.124239 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:50:25.124186    2576 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 14 07:50:25.124314 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:50:25.124276    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725b3545-1cdb-48bf-8a09-d20468e44525-prometheus-operator-tls podName:725b3545-1cdb-48bf-8a09-d20468e44525 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:25.624260148 +0000 UTC m=+80.328798632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/725b3545-1cdb-48bf-8a09-d20468e44525-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-b4bg7" (UID: "725b3545-1cdb-48bf-8a09-d20468e44525") : secret "prometheus-operator-tls" not found
Sep 14 07:50:25.124680 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.124653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/725b3545-1cdb-48bf-8a09-d20468e44525-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-b4bg7\" (UID: \"725b3545-1cdb-48bf-8a09-d20468e44525\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:25.126550 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.126529    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/725b3545-1cdb-48bf-8a09-d20468e44525-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-b4bg7\" (UID: \"725b3545-1cdb-48bf-8a09-d20468e44525\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:25.134450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.134421    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nvpq\" (UniqueName: \"kubernetes.io/projected/725b3545-1cdb-48bf-8a09-d20468e44525-kube-api-access-8nvpq\") pod \"prometheus-operator-68bcd7f987-b4bg7\" (UID: \"725b3545-1cdb-48bf-8a09-d20468e44525\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:25.269725 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.269668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:25.627166 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.627140    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/725b3545-1cdb-48bf-8a09-d20468e44525-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-b4bg7\" (UID: \"725b3545-1cdb-48bf-8a09-d20468e44525\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:25.630116 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.630095    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/725b3545-1cdb-48bf-8a09-d20468e44525-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-b4bg7\" (UID: \"725b3545-1cdb-48bf-8a09-d20468e44525\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:25.834875 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:25.834844    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"
Sep 14 07:50:26.107180 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:26.107146    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7"]
Sep 14 07:50:26.110264 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:50:26.110235    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod725b3545_1cdb_48bf_8a09_d20468e44525.slice/crio-28f3f21be50a08300a303a83bf2033a3998e4a29ad079d6dda1f4ab323d13010 WatchSource:0}: Error finding container 28f3f21be50a08300a303a83bf2033a3998e4a29ad079d6dda1f4ab323d13010: Status 404 returned error can't find the container with id 28f3f21be50a08300a303a83bf2033a3998e4a29ad079d6dda1f4ab323d13010
Sep 14 07:50:26.190001 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:26.189937    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7" event={"ID":"725b3545-1cdb-48bf-8a09-d20468e44525","Type":"ContainerStarted","Data":"28f3f21be50a08300a303a83bf2033a3998e4a29ad079d6dda1f4ab323d13010"}
Sep 14 07:50:27.455694 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:27.455645    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:28.189942 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:28.189818    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:28.198952 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:28.198918    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7" event={"ID":"725b3545-1cdb-48bf-8a09-d20468e44525","Type":"ContainerStarted","Data":"7d2afc8cc167944cf977a4f96b3c7ee3d474bc31c09c4a905a6ec0d59d6facbb"}
Sep 14 07:50:28.199063 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:28.198971    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7" event={"ID":"725b3545-1cdb-48bf-8a09-d20468e44525","Type":"ContainerStarted","Data":"c1f36fa084224682f5cecc43a71017f7452ac5a5d9811af78f9a13957569461e"}
Sep 14 07:50:28.214455 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:28.214403    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-b4bg7" podStartSLOduration=3.118950095 podStartE2EDuration="4.214386416s" totalImagesPullingTime="1.095436321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:28.213648958 +0000 UTC m=+82.918187484" watchObservedRunningTime="2026-09-14 07:50:28.214386416 +0000 UTC m=+82.918924918"
Sep 14 07:50:28.291223 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:28.291174    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:30.008296 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.008265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:30.176254 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.176230    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pp6kf"
Sep 14 07:50:30.227883 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.227849    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:30.276546 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.276478    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"]
Sep 14 07:50:30.278784 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.278763    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.280901 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.280884    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 14 07:50:30.281040 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.281021    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 14 07:50:30.281193 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.281173    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 14 07:50:30.281338 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.281250    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-p7xj4\""
Sep 14 07:50:30.290504 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.290486    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"]
Sep 14 07:50:30.311296 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.311274    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hkkt9"]
Sep 14 07:50:30.314166 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.314143    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.316463 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.316446    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 14 07:50:30.316569 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.316476    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 14 07:50:30.316795 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.316775    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bcz9h\""
Sep 14 07:50:30.316850 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.316804    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 14 07:50:30.320382 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.320355    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:30.359068 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.359046    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.359158 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.359097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.359158 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.359146    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.359282 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.359162    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.359282 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.359178    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.359282 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.359220    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw6h8\" (UniqueName: \"kubernetes.io/projected/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-kube-api-access-mw6h8\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.382498 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.382477    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:30.449462 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.449442    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:30.459738 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.459716    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.459819 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.459742    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.459819 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.459758    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.459819 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.459777    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12caa7c9-5dfe-4da2-840e-227c707440e9-sys\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.459819 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.459800    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12caa7c9-5dfe-4da2-840e-227c707440e9-metrics-client-ca\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.460006 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.459824    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.460006 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.459890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12caa7c9-5dfe-4da2-840e-227c707440e9-root\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.460006 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.459933    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw6h8\" (UniqueName: \"kubernetes.io/projected/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-kube-api-access-mw6h8\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.460006 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.459954    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-accelerators-collector-config\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.460006 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.459985    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9wfv\" (UniqueName: \"kubernetes.io/projected/12caa7c9-5dfe-4da2-840e-227c707440e9-kube-api-access-k9wfv\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.460273 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.460015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.460273 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.460042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-wtmp\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.460273 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.460151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.460273 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.460162    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-tls\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.460419 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.460266    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.460419 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.460332    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-textfile\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.460610 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.460590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.460695 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.460678    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.462312 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.462292    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.462552 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.462533    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.469730 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.469708    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw6h8\" (UniqueName: \"kubernetes.io/projected/58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b-kube-api-access-mw6h8\") pod \"kube-state-metrics-59fbdd77d-2mq8j\" (UID: \"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.550103 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.550047    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:30.561084 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.561053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9wfv\" (UniqueName: \"kubernetes.io/projected/12caa7c9-5dfe-4da2-840e-227c707440e9-kube-api-access-k9wfv\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.561196 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.561088    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-wtmp\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.561196 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.561122    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-tls\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.561334 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.561170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-textfile\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.561334 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.561259    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12caa7c9-5dfe-4da2-840e-227c707440e9-sys\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.561334 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.561279    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12caa7c9-5dfe-4da2-840e-227c707440e9-metrics-client-ca\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.561334 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.561304    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.561334 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.561331    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12caa7c9-5dfe-4da2-840e-227c707440e9-root\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.561567 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.561360    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-wtmp\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.561567 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.561454    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12caa7c9-5dfe-4da2-840e-227c707440e9-sys\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.561567 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.561507    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12caa7c9-5dfe-4da2-840e-227c707440e9-root\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.561567 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.561364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-accelerators-collector-config\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.561774 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.561576    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-textfile\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.562032 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.562010    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-accelerators-collector-config\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.562032 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.562025    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12caa7c9-5dfe-4da2-840e-227c707440e9-metrics-client-ca\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.563407 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.563388    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.563578 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.563561    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12caa7c9-5dfe-4da2-840e-227c707440e9-node-exporter-tls\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.568534 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.568513    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9wfv\" (UniqueName: \"kubernetes.io/projected/12caa7c9-5dfe-4da2-840e-227c707440e9-kube-api-access-k9wfv\") pod \"node-exporter-hkkt9\" (UID: \"12caa7c9-5dfe-4da2-840e-227c707440e9\") " pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.588372 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.588352    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"
Sep 14 07:50:30.622936 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.622908    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hkkt9"
Sep 14 07:50:30.634104 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:50:30.634047    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12caa7c9_5dfe_4da2_840e_227c707440e9.slice/crio-98d13706ac4559523009215c8c1033383734117822a5a15895bdba4e18774ffb WatchSource:0}: Error finding container 98d13706ac4559523009215c8c1033383734117822a5a15895bdba4e18774ffb: Status 404 returned error can't find the container with id 98d13706ac4559523009215c8c1033383734117822a5a15895bdba4e18774ffb
Sep 14 07:50:30.651122 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.651070    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:30.862072 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:30.862050    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j"]
Sep 14 07:50:30.865106 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:50:30.865079    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58cd67d9_571d_4ea3_bfa3_ccc5bf14e75b.slice/crio-b403f97ee08c22d9995b302887de43adade22e8e3fec18066bc39770dacf9453 WatchSource:0}: Error finding container b403f97ee08c22d9995b302887de43adade22e8e3fec18066bc39770dacf9453: Status 404 returned error can't find the container with id b403f97ee08c22d9995b302887de43adade22e8e3fec18066bc39770dacf9453
Sep 14 07:50:31.211068 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:31.210992    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j" event={"ID":"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b","Type":"ContainerStarted","Data":"b403f97ee08c22d9995b302887de43adade22e8e3fec18066bc39770dacf9453"}
Sep 14 07:50:31.212271 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:31.212247    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hkkt9" event={"ID":"12caa7c9-5dfe-4da2-840e-227c707440e9","Type":"ContainerStarted","Data":"98d13706ac4559523009215c8c1033383734117822a5a15895bdba4e18774ffb"}
Sep 14 07:50:31.288476 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:31.288445    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:31.368381 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:31.368340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:31.677072 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:31.677033    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:31.867451 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:31.867418    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:32.218108 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:32.218064    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j" event={"ID":"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b","Type":"ContainerStarted","Data":"a64d6c09238056b3fd128c1fe08618b54d6d77368cdbaab560ab7066f6a37091"}
Sep 14 07:50:32.220075 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:32.220050    2576 generic.go:356] "Generic (PLEG): container finished" podID="12caa7c9-5dfe-4da2-840e-227c707440e9" containerID="14055e16d173be8821cce451bdec80e3dc02105bcf742c8b849fc13481266120" exitCode=0
Sep 14 07:50:32.220163 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:32.220090    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hkkt9" event={"ID":"12caa7c9-5dfe-4da2-840e-227c707440e9","Type":"ContainerDied","Data":"14055e16d173be8821cce451bdec80e3dc02105bcf742c8b849fc13481266120"}
Sep 14 07:50:32.540736 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:32.540662    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:32.605943 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:32.605909    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:33.172605 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:33.172578    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:33.226405 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:33.226371    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j" event={"ID":"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b","Type":"ContainerStarted","Data":"8a2e3fe40020da8cfaceceafc13644d843a64b284fc2c71f4d5ca3a6bb45e6d6"}
Sep 14 07:50:33.226758 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:33.226418    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j" event={"ID":"58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b","Type":"ContainerStarted","Data":"7d76e14b53a34bc327356451968c51c64b88108713bd0aa8a3acc27ed4650687"}
Sep 14 07:50:33.228594 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:33.228567    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hkkt9" event={"ID":"12caa7c9-5dfe-4da2-840e-227c707440e9","Type":"ContainerStarted","Data":"ff877833f20e69456313de31130877f028d6d40060d32e878f700fa05cadd5d0"}
Sep 14 07:50:33.228693 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:33.228622    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hkkt9" event={"ID":"12caa7c9-5dfe-4da2-840e-227c707440e9","Type":"ContainerStarted","Data":"ec82638ac2e84bdf8271524c0214ca5c19c058487df87f617a04436b3574ac0f"}
Sep 14 07:50:33.250828 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:33.250805    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:33.265441 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:33.265406    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hkkt9" podStartSLOduration=2.268078108 podStartE2EDuration="3.265391166s" totalImagesPullingTime="925.448904ms" totalInitContainerRuntime="71.864154ms" isStatefulPod=true podCreationTimestamp="2026-09-14 07:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:33.264287534 +0000 UTC m=+87.968826038" watchObservedRunningTime="2026-09-14 07:50:33.265391166 +0000 UTC m=+87.969929667"
Sep 14 07:50:33.265662 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:33.265643    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2mq8j" podStartSLOduration=2.010487264 podStartE2EDuration="3.265639523s" totalImagesPullingTime="1.255152259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:33.247282145 +0000 UTC m=+87.951820647" watchObservedRunningTime="2026-09-14 07:50:33.265639523 +0000 UTC m=+87.970178025"
Sep 14 07:50:33.327538 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:33.327519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:33.427759 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:33.427708    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:33.652475 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:33.652447    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:33.752769 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:33.752702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:34.413831 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:34.413801    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:34.477178 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:34.477155    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:35.062937 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.062909    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:35.074771 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.074741    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68"]
Sep 14 07:50:35.081135 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.081114    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68"
Sep 14 07:50:35.083303 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.083278    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 14 07:50:35.083543 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.083527    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-cnbbx\""
Sep 14 07:50:35.085973 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.085952    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68"]
Sep 14 07:50:35.144681 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.144656    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:35.195481 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.195451    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b1667a48-f833-4284-b7c3-c8fadef6b81d-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-9qc68\" (UID: \"b1667a48-f833-4284-b7c3-c8fadef6b81d\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68"
Sep 14 07:50:35.195579 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.195503    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49n68\" (UniqueName: \"kubernetes.io/projected/b1667a48-f833-4284-b7c3-c8fadef6b81d-kube-api-access-49n68\") pod \"monitoring-plugin-5d85778b9d-9qc68\" (UID: \"b1667a48-f833-4284-b7c3-c8fadef6b81d\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68"
Sep 14 07:50:35.296476 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.296442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b1667a48-f833-4284-b7c3-c8fadef6b81d-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-9qc68\" (UID: \"b1667a48-f833-4284-b7c3-c8fadef6b81d\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68"
Sep 14 07:50:35.296629 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.296506    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49n68\" (UniqueName: \"kubernetes.io/projected/b1667a48-f833-4284-b7c3-c8fadef6b81d-kube-api-access-49n68\") pod \"monitoring-plugin-5d85778b9d-9qc68\" (UID: \"b1667a48-f833-4284-b7c3-c8fadef6b81d\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68"
Sep 14 07:50:35.296629 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:50:35.296604    2576 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 14 07:50:35.296748 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:50:35.296675    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1667a48-f833-4284-b7c3-c8fadef6b81d-monitoring-plugin-cert podName:b1667a48-f833-4284-b7c3-c8fadef6b81d nodeName:}" failed. No retries permitted until 2026-09-14 07:50:35.796659017 +0000 UTC m=+90.501197500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b1667a48-f833-4284-b7c3-c8fadef6b81d-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-9qc68" (UID: "b1667a48-f833-4284-b7c3-c8fadef6b81d") : secret "monitoring-plugin-cert" not found
Sep 14 07:50:35.304800 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.304769    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49n68\" (UniqueName: \"kubernetes.io/projected/b1667a48-f833-4284-b7c3-c8fadef6b81d-kube-api-access-49n68\") pod \"monitoring-plugin-5d85778b9d-9qc68\" (UID: \"b1667a48-f833-4284-b7c3-c8fadef6b81d\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68"
Sep 14 07:50:35.501660 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.501628    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:35.800335 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.800277    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b1667a48-f833-4284-b7c3-c8fadef6b81d-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-9qc68\" (UID: \"b1667a48-f833-4284-b7c3-c8fadef6b81d\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68"
Sep 14 07:50:35.802517 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.802497    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b1667a48-f833-4284-b7c3-c8fadef6b81d-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-9qc68\" (UID: \"b1667a48-f833-4284-b7c3-c8fadef6b81d\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68"
Sep 14 07:50:35.992061 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:35.992042    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68"
Sep 14 07:50:36.148259 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:36.148237    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7bc5c4ffc4-p2pqp"
Sep 14 07:50:36.259949 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:36.259918    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68"]
Sep 14 07:50:36.263071 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:50:36.263046    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1667a48_f833_4284_b7c3_c8fadef6b81d.slice/crio-207bc6b83c04763a81e9758c6937f038caed61cf591ea40e5c4c96e594d56798 WatchSource:0}: Error finding container 207bc6b83c04763a81e9758c6937f038caed61cf591ea40e5c4c96e594d56798: Status 404 returned error can't find the container with id 207bc6b83c04763a81e9758c6937f038caed61cf591ea40e5c4c96e594d56798
Sep 14 07:50:36.342330 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:36.342232    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:36.439907 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:36.439817    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:36.541395 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:36.541360    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:37.247043 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:37.247009    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68" event={"ID":"b1667a48-f833-4284-b7c3-c8fadef6b81d","Type":"ContainerStarted","Data":"207bc6b83c04763a81e9758c6937f038caed61cf591ea40e5c4c96e594d56798"}
Sep 14 07:50:38.251761 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:38.251732    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68" event={"ID":"b1667a48-f833-4284-b7c3-c8fadef6b81d","Type":"ContainerStarted","Data":"ce181adee283a4d5057fe826fe9facd6afd8ff0526c2d27d2709b047e00c6fcb"}
Sep 14 07:50:38.252110 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:38.251915    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68"
Sep 14 07:50:38.257068 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:38.257045    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68"
Sep 14 07:50:38.267270 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:38.267223    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-9qc68" podStartSLOduration=1.452154088 podStartE2EDuration="3.267192422s" totalImagesPullingTime="1.815038334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:38.265577292 +0000 UTC m=+92.970115836" watchObservedRunningTime="2026-09-14 07:50:38.267192422 +0000 UTC m=+92.971730924"
Sep 14 07:50:38.404569 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:38.404539    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:38.506447 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:38.506338    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:40.816827 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:40.816771    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:40.867450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:40.867415    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:41.546561 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:41.546529    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:44.137792 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:44.137749    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:44.236905 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:44.236862    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:44.337763 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:44.337738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:45.697591 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:45.697557    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:45.807060 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:45.807007    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:45.898037 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:45.897993    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:49.884778 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:49.884742    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:49.941471 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:49.941445    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:51.176290 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:51.176254    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qjv9w"
Sep 14 07:50:51.265856 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:51.265827    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:51.364222 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:51.364182    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:51.464372 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:51.464308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:51.826774 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:50:51.826715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:14.175196 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:14.175166    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:14.276276 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:14.276235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:14.877965 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:14.877933    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:14.935283 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:14.935238    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:18.405081 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:18.405054    2576 generic.go:356] "Generic (PLEG): container finished" podID="c4e8db51-9e25-4dff-be31-b15688ddf1be" containerID="491b9df3d96541627fb8bc43764fdf0dffce05fff83bfb9df1091df2c13f46fb" exitCode=0
Sep 14 07:51:18.405409 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:18.405096    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5" event={"ID":"c4e8db51-9e25-4dff-be31-b15688ddf1be","Type":"ContainerDied","Data":"491b9df3d96541627fb8bc43764fdf0dffce05fff83bfb9df1091df2c13f46fb"}
Sep 14 07:51:18.405409 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:18.405400    2576 scope.go:117] "RemoveContainer" containerID="491b9df3d96541627fb8bc43764fdf0dffce05fff83bfb9df1091df2c13f46fb"
Sep 14 07:51:19.411423 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:19.411380    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-29mn5" event={"ID":"c4e8db51-9e25-4dff-be31-b15688ddf1be","Type":"ContainerStarted","Data":"01a0f3b9cadf633d4f26f065c0d05a9d475ecb8a68c389f253a04e8af92d1cdc"}
Sep 14 07:51:20.270917 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:20.270869    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:20.333645 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:20.333608    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:44.880358 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:44.880320    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:49.448273 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:49.448186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:49.656757 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:49.656728    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:57.871033 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:57.870951    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:57.923457 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:51:57.923431    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:04.715987 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:52:04.715955    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:04.802388 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:52:04.802354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:16.667323 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:52:16.667277    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:16.725589 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:52:16.725565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:17.024029 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:52:17.023960    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:17.126461 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:52:17.126427    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:17.182625 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:52:17.182598    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:29.181384 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:52:29.181345    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:29.282543 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:52:29.282501    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:50.513559 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:52:50.513521    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:50.571043 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:52:50.571009    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:57.724779 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:52:57.724738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:30.894533 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:53:30.894433    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:30.990203 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:53:30.990175    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:57.649338 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:53:57.649300    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:58.432706 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:53:58.432671    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:01.257638 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:01.257605    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:01.358138 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:01.358107    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:02.694120 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:02.694089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:02.847297 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:02.847267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:03.676353 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.676321    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"]
Sep 14 07:54:03.679796 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.679776    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"
Sep 14 07:54:03.682198 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.682171    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 07:54:03.682569 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.682543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f460d6c-e7b9-489e-ace4-77b395536548-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww\" (UID: \"7f460d6c-e7b9-489e-ace4-77b395536548\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"
Sep 14 07:54:03.682644 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.682585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjhqw\" (UniqueName: \"kubernetes.io/projected/7f460d6c-e7b9-489e-ace4-77b395536548-kube-api-access-hjhqw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww\" (UID: \"7f460d6c-e7b9-489e-ace4-77b395536548\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"
Sep 14 07:54:03.682644 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.682616    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f460d6c-e7b9-489e-ace4-77b395536548-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww\" (UID: \"7f460d6c-e7b9-489e-ace4-77b395536548\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"
Sep 14 07:54:03.683229 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.683192    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 07:54:03.683315 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.683243    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9nkq4\""
Sep 14 07:54:03.688067 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.688043    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"]
Sep 14 07:54:03.782877 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.782851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f460d6c-e7b9-489e-ace4-77b395536548-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww\" (UID: \"7f460d6c-e7b9-489e-ace4-77b395536548\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"
Sep 14 07:54:03.783172 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.782888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjhqw\" (UniqueName: \"kubernetes.io/projected/7f460d6c-e7b9-489e-ace4-77b395536548-kube-api-access-hjhqw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww\" (UID: \"7f460d6c-e7b9-489e-ace4-77b395536548\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"
Sep 14 07:54:03.783172 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.782912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f460d6c-e7b9-489e-ace4-77b395536548-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww\" (UID: \"7f460d6c-e7b9-489e-ace4-77b395536548\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"
Sep 14 07:54:03.783271 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.783244    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f460d6c-e7b9-489e-ace4-77b395536548-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww\" (UID: \"7f460d6c-e7b9-489e-ace4-77b395536548\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"
Sep 14 07:54:03.783321 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.783301    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f460d6c-e7b9-489e-ace4-77b395536548-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww\" (UID: \"7f460d6c-e7b9-489e-ace4-77b395536548\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"
Sep 14 07:54:03.790499 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.790470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjhqw\" (UniqueName: \"kubernetes.io/projected/7f460d6c-e7b9-489e-ace4-77b395536548-kube-api-access-hjhqw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww\" (UID: \"7f460d6c-e7b9-489e-ace4-77b395536548\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"
Sep 14 07:54:03.990608 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:03.990545    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"
Sep 14 07:54:04.263196 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:04.263169    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"]
Sep 14 07:54:04.265750 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:54:04.265722    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f460d6c_e7b9_489e_ace4_77b395536548.slice/crio-ef0cbea5b774efaf366f9bf178bb361dfcd4cc6b2ee108567eaf5d49ddbc93b2 WatchSource:0}: Error finding container ef0cbea5b774efaf366f9bf178bb361dfcd4cc6b2ee108567eaf5d49ddbc93b2: Status 404 returned error can't find the container with id ef0cbea5b774efaf366f9bf178bb361dfcd4cc6b2ee108567eaf5d49ddbc93b2
Sep 14 07:54:05.021781 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:05.021730    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww" event={"ID":"7f460d6c-e7b9-489e-ace4-77b395536548","Type":"ContainerStarted","Data":"ef0cbea5b774efaf366f9bf178bb361dfcd4cc6b2ee108567eaf5d49ddbc93b2"}
Sep 14 07:54:05.048993 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:05.048967    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:05.499220 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:05.499177    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:05.731158 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:05.731128    2576 kubelet.go:1648] "Image garbage collection succeeded"
Sep 14 07:54:07.882149 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:07.882118    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qr8hm"]
Sep 14 07:54:07.885146 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:07.885131    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qr8hm"
Sep 14 07:54:07.887503 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:07.887479    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 14 07:54:07.887503 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:07.887493    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 14 07:54:07.887727 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:07.887709    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8tg8m\""
Sep 14 07:54:07.895309 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:07.895285    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qr8hm"]
Sep 14 07:54:07.912769 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:07.912747    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mblnp\" (UniqueName: \"kubernetes.io/projected/70605861-5193-4afb-ab85-0398a7180ef4-kube-api-access-mblnp\") pod \"cert-manager-cainjector-559cdd8c87-qr8hm\" (UID: \"70605861-5193-4afb-ab85-0398a7180ef4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qr8hm"
Sep 14 07:54:07.912879 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:07.912804    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70605861-5193-4afb-ab85-0398a7180ef4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qr8hm\" (UID: \"70605861-5193-4afb-ab85-0398a7180ef4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qr8hm"
Sep 14 07:54:08.014030 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:08.014004    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70605861-5193-4afb-ab85-0398a7180ef4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qr8hm\" (UID: \"70605861-5193-4afb-ab85-0398a7180ef4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qr8hm"
Sep 14 07:54:08.014144 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:08.014066    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mblnp\" (UniqueName: \"kubernetes.io/projected/70605861-5193-4afb-ab85-0398a7180ef4-kube-api-access-mblnp\") pod \"cert-manager-cainjector-559cdd8c87-qr8hm\" (UID: \"70605861-5193-4afb-ab85-0398a7180ef4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qr8hm"
Sep 14 07:54:08.025187 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:08.025150    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70605861-5193-4afb-ab85-0398a7180ef4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qr8hm\" (UID: \"70605861-5193-4afb-ab85-0398a7180ef4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qr8hm"
Sep 14 07:54:08.025576 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:08.025555    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mblnp\" (UniqueName: \"kubernetes.io/projected/70605861-5193-4afb-ab85-0398a7180ef4-kube-api-access-mblnp\") pod \"cert-manager-cainjector-559cdd8c87-qr8hm\" (UID: \"70605861-5193-4afb-ab85-0398a7180ef4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qr8hm"
Sep 14 07:54:08.050035 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:08.050005    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:08.195200 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:08.195141    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qr8hm"
Sep 14 07:54:08.473310 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:08.473286    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qr8hm"]
Sep 14 07:54:09.257315 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:09.257254    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:10.222836 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:54:10.222800    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70605861_5193_4afb_ab85_0398a7180ef4.slice/crio-38b147d87e3ad21840c8fa208e0bed318832f83f6594c003d82f97c0262b410f WatchSource:0}: Error finding container 38b147d87e3ad21840c8fa208e0bed318832f83f6594c003d82f97c0262b410f: Status 404 returned error can't find the container with id 38b147d87e3ad21840c8fa208e0bed318832f83f6594c003d82f97c0262b410f
Sep 14 07:54:10.224895 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:10.224878    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:54:11.049008 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:11.048948    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qr8hm" event={"ID":"70605861-5193-4afb-ab85-0398a7180ef4","Type":"ContainerStarted","Data":"38b147d87e3ad21840c8fa208e0bed318832f83f6594c003d82f97c0262b410f"}
Sep 14 07:54:11.051873 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:11.051836    2576 generic.go:356] "Generic (PLEG): container finished" podID="7f460d6c-e7b9-489e-ace4-77b395536548" containerID="61e8de1d56d81d9f3a1e4d6a2bfe4c103949af3079882fd9104e322e41168f75" exitCode=0
Sep 14 07:54:11.052030 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:11.051916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww" event={"ID":"7f460d6c-e7b9-489e-ace4-77b395536548","Type":"ContainerDied","Data":"61e8de1d56d81d9f3a1e4d6a2bfe4c103949af3079882fd9104e322e41168f75"}
Sep 14 07:54:12.655916 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:12.655879    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:13.968580 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:13.968550    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:14.029742 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:14.029717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:14.064297 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:14.064270    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qr8hm" event={"ID":"70605861-5193-4afb-ab85-0398a7180ef4","Type":"ContainerStarted","Data":"ca53f757e795822e6609f85be9bdaa03a68cd58645b60b5a08784b14db203bba"}
Sep 14 07:54:14.096664 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:14.096617    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qr8hm" podStartSLOduration=4.132867182 podStartE2EDuration="7.09657753s" totalImagesPullingTime="2.963710348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:54:14.094676322 +0000 UTC m=+308.799214825" watchObservedRunningTime="2026-09-14 07:54:14.09657753 +0000 UTC m=+308.801116031"
Sep 14 07:54:31.853692 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:31.853660    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:31.907165 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:31.907139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:32.133124 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:32.133052    2576 generic.go:356] "Generic (PLEG): container finished" podID="7f460d6c-e7b9-489e-ace4-77b395536548" containerID="f17997f31bb7b96eda5ef2bec078dec9b79e7312145b17c6a2014c47f1109b9a" exitCode=0
Sep 14 07:54:32.133263 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:32.133141    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww" event={"ID":"7f460d6c-e7b9-489e-ace4-77b395536548","Type":"ContainerDied","Data":"f17997f31bb7b96eda5ef2bec078dec9b79e7312145b17c6a2014c47f1109b9a"}
Sep 14 07:54:39.164291 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:39.164237    2576 generic.go:356] "Generic (PLEG): container finished" podID="7f460d6c-e7b9-489e-ace4-77b395536548" containerID="d83090bc34ee13282a05caeb0094748dab6752871dbaa03879b11e9797763f7c" exitCode=0
Sep 14 07:54:39.164663 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:39.164328    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww" event={"ID":"7f460d6c-e7b9-489e-ace4-77b395536548","Type":"ContainerDied","Data":"d83090bc34ee13282a05caeb0094748dab6752871dbaa03879b11e9797763f7c"}
Sep 14 07:54:40.298450 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:40.298426    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"
Sep 14 07:54:40.346658 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:40.346639    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f460d6c-e7b9-489e-ace4-77b395536548-util\") pod \"7f460d6c-e7b9-489e-ace4-77b395536548\" (UID: \"7f460d6c-e7b9-489e-ace4-77b395536548\") "
Sep 14 07:54:40.346770 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:40.346677    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjhqw\" (UniqueName: \"kubernetes.io/projected/7f460d6c-e7b9-489e-ace4-77b395536548-kube-api-access-hjhqw\") pod \"7f460d6c-e7b9-489e-ace4-77b395536548\" (UID: \"7f460d6c-e7b9-489e-ace4-77b395536548\") "
Sep 14 07:54:40.346770 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:40.346693    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f460d6c-e7b9-489e-ace4-77b395536548-bundle\") pod \"7f460d6c-e7b9-489e-ace4-77b395536548\" (UID: \"7f460d6c-e7b9-489e-ace4-77b395536548\") "
Sep 14 07:54:40.347079 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:40.347057    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f460d6c-e7b9-489e-ace4-77b395536548-bundle" (OuterVolumeSpecName: "bundle") pod "7f460d6c-e7b9-489e-ace4-77b395536548" (UID: "7f460d6c-e7b9-489e-ace4-77b395536548"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:54:40.348771 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:40.348743    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f460d6c-e7b9-489e-ace4-77b395536548-kube-api-access-hjhqw" (OuterVolumeSpecName: "kube-api-access-hjhqw") pod "7f460d6c-e7b9-489e-ace4-77b395536548" (UID: "7f460d6c-e7b9-489e-ace4-77b395536548"). InnerVolumeSpecName "kube-api-access-hjhqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:54:40.350908 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:40.350891    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f460d6c-e7b9-489e-ace4-77b395536548-util" (OuterVolumeSpecName: "util") pod "7f460d6c-e7b9-489e-ace4-77b395536548" (UID: "7f460d6c-e7b9-489e-ace4-77b395536548"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:54:40.447183 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:40.447123    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f460d6c-e7b9-489e-ace4-77b395536548-util\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 07:54:40.447183 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:40.447154    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjhqw\" (UniqueName: \"kubernetes.io/projected/7f460d6c-e7b9-489e-ace4-77b395536548-kube-api-access-hjhqw\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 07:54:40.447183 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:40.447165    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f460d6c-e7b9-489e-ace4-77b395536548-bundle\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 07:54:41.173429 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:41.173402    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww"
Sep 14 07:54:41.173429 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:41.173421    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmbtww" event={"ID":"7f460d6c-e7b9-489e-ace4-77b395536548","Type":"ContainerDied","Data":"ef0cbea5b774efaf366f9bf178bb361dfcd4cc6b2ee108567eaf5d49ddbc93b2"}
Sep 14 07:54:41.173591 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:41.173460    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef0cbea5b774efaf366f9bf178bb361dfcd4cc6b2ee108567eaf5d49ddbc93b2"
Sep 14 07:54:45.687230 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:45.687183    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:45.785868 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:45.785837    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:51.191321 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:51.191285    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:51.292623 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:51.292591    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:53.242831 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:53.242782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:55.577086 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:55.577018    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:57.392430 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:57.392395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:57.469807 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:54:57.469777    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:08.841065 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:08.841036    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:08.940030 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:08.940005    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:09.650265 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.650233    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:09.871113 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.871084    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:09.974890 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.974816    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:09.987220 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.987180    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"]
Sep 14 07:55:09.987468 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.987453    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f460d6c-e7b9-489e-ace4-77b395536548" containerName="extract"
Sep 14 07:55:09.987509 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.987469    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f460d6c-e7b9-489e-ace4-77b395536548" containerName="extract"
Sep 14 07:55:09.987509 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.987482    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f460d6c-e7b9-489e-ace4-77b395536548" containerName="pull"
Sep 14 07:55:09.987509 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.987488    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f460d6c-e7b9-489e-ace4-77b395536548" containerName="pull"
Sep 14 07:55:09.987509 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.987497    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f460d6c-e7b9-489e-ace4-77b395536548" containerName="util"
Sep 14 07:55:09.987509 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.987502    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f460d6c-e7b9-489e-ace4-77b395536548" containerName="util"
Sep 14 07:55:09.987653 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.987545    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f460d6c-e7b9-489e-ace4-77b395536548" containerName="extract"
Sep 14 07:55:09.994939 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.994919    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:09.997871 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.997854    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 14 07:55:09.998154 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.998134    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 14 07:55:09.998233 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.998137    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 14 07:55:09.998296 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.998280    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-b4wwj\""
Sep 14 07:55:09.998561 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:09.998545    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 14 07:55:10.016488 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.016466    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"]
Sep 14 07:55:10.157267 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.157240    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df73aa99-837f-4751-9b29-6143cf8ef9be-manifests\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.157356 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.157278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df73aa99-837f-4751-9b29-6143cf8ef9be-tmp\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.157356 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.157323    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df73aa99-837f-4751-9b29-6143cf8ef9be-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.157356 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.157349    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnk95\" (UniqueName: \"kubernetes.io/projected/df73aa99-837f-4751-9b29-6143cf8ef9be-kube-api-access-xnk95\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.157489 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.157446    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df73aa99-837f-4751-9b29-6143cf8ef9be-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.192956 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.192931    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:10.258072 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.258019    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df73aa99-837f-4751-9b29-6143cf8ef9be-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.258072 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.258053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df73aa99-837f-4751-9b29-6143cf8ef9be-manifests\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.258072 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.258072    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df73aa99-837f-4751-9b29-6143cf8ef9be-tmp\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.258292 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.258087    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df73aa99-837f-4751-9b29-6143cf8ef9be-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.258292 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.258108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnk95\" (UniqueName: \"kubernetes.io/projected/df73aa99-837f-4751-9b29-6143cf8ef9be-kube-api-access-xnk95\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.258547 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.258524    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df73aa99-837f-4751-9b29-6143cf8ef9be-manifests\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.260591 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.260565    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df73aa99-837f-4751-9b29-6143cf8ef9be-tmp\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.260695 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.260660    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df73aa99-837f-4751-9b29-6143cf8ef9be-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.260775 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.260755    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df73aa99-837f-4751-9b29-6143cf8ef9be-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.265524 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.265502    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnk95\" (UniqueName: \"kubernetes.io/projected/df73aa99-837f-4751-9b29-6143cf8ef9be-kube-api-access-xnk95\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-qxz66\" (UID: \"df73aa99-837f-4751-9b29-6143cf8ef9be\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.298294 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.298268    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:10.305253 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.305234    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:10.394106 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.394078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:10.491175 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.491142    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:10.561150 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.561095    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:10.577721 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:10.577701    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"]
Sep 14 07:55:10.580159 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:55:10.580131    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf73aa99_837f_4751_9b29_6143cf8ef9be.slice/crio-2de4cacc009b39a95855a35189e39c647e1090ccfc31ef5e09ae2dd8de62b606 WatchSource:0}: Error finding container 2de4cacc009b39a95855a35189e39c647e1090ccfc31ef5e09ae2dd8de62b606: Status 404 returned error can't find the container with id 2de4cacc009b39a95855a35189e39c647e1090ccfc31ef5e09ae2dd8de62b606
Sep 14 07:55:11.290085 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:11.290042    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66" event={"ID":"df73aa99-837f-4751-9b29-6143cf8ef9be","Type":"ContainerStarted","Data":"2de4cacc009b39a95855a35189e39c647e1090ccfc31ef5e09ae2dd8de62b606"}
Sep 14 07:55:14.305639 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:14.305567    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66" event={"ID":"df73aa99-837f-4751-9b29-6143cf8ef9be","Type":"ContainerStarted","Data":"04c466eb5024284c646a956109150df3f484a8015d3c58f2e9710feb1a54ad7c"}
Sep 14 07:55:15.311499 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:15.311467    2576 generic.go:356] "Generic (PLEG): container finished" podID="df73aa99-837f-4751-9b29-6143cf8ef9be" containerID="04c466eb5024284c646a956109150df3f484a8015d3c58f2e9710feb1a54ad7c" exitCode=0
Sep 14 07:55:15.311802 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:15.311524    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66" event={"ID":"df73aa99-837f-4751-9b29-6143cf8ef9be","Type":"ContainerDied","Data":"04c466eb5024284c646a956109150df3f484a8015d3c58f2e9710feb1a54ad7c"}
Sep 14 07:55:16.317327 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:16.317297    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66" event={"ID":"df73aa99-837f-4751-9b29-6143cf8ef9be","Type":"ContainerStarted","Data":"c4d50a7f67be62e887a955f713df55c0f37e323497d01a1d295372c980dab06c"}
Sep 14 07:55:16.317720 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:16.317408    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:16.338151 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:16.338091    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66" podStartSLOduration=2.499136742 podStartE2EDuration="7.338072942s" totalImagesPullingTime="3.797032377s" totalInitContainerRuntime="1.041903823s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:16.336891058 +0000 UTC m=+371.041429559" watchObservedRunningTime="2026-09-14 07:55:16.338072942 +0000 UTC m=+371.042611451"
Sep 14 07:55:25.047543 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:25.047508    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:25.192877 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:25.192848    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:25.785554 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:25.785518    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:25.886590 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:25.886562    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:27.327272 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:27.327233    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-qxz66"
Sep 14 07:55:29.447854 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:29.447818    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:29.554004 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:29.553966    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:30.006434 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.006401    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:30.053078 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.053050    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h"]
Sep 14 07:55:30.056035 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.056018    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h"
Sep 14 07:55:30.058975 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.058953    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 14 07:55:30.059065 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.058985    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 14 07:55:30.059939 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.059923    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qzx2g\""
Sep 14 07:55:30.064936 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.064915    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h"]
Sep 14 07:55:30.093823 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.093793    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:30.196275 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.196236    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l546f\" (UniqueName: \"kubernetes.io/projected/e792ccde-27e1-4543-8252-0c5468bab8dc-kube-api-access-l546f\") pod \"kube-auth-proxy-75fb8dbff8-bfs7h\" (UID: \"e792ccde-27e1-4543-8252-0c5468bab8dc\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h"
Sep 14 07:55:30.196374 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.196281    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e792ccde-27e1-4543-8252-0c5468bab8dc-tmp\") pod \"kube-auth-proxy-75fb8dbff8-bfs7h\" (UID: \"e792ccde-27e1-4543-8252-0c5468bab8dc\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h"
Sep 14 07:55:30.196374 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.196324    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e792ccde-27e1-4543-8252-0c5468bab8dc-tls-certs\") pod \"kube-auth-proxy-75fb8dbff8-bfs7h\" (UID: \"e792ccde-27e1-4543-8252-0c5468bab8dc\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h"
Sep 14 07:55:30.297169 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.297116    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l546f\" (UniqueName: \"kubernetes.io/projected/e792ccde-27e1-4543-8252-0c5468bab8dc-kube-api-access-l546f\") pod \"kube-auth-proxy-75fb8dbff8-bfs7h\" (UID: \"e792ccde-27e1-4543-8252-0c5468bab8dc\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h"
Sep 14 07:55:30.297169 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.297145    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e792ccde-27e1-4543-8252-0c5468bab8dc-tmp\") pod \"kube-auth-proxy-75fb8dbff8-bfs7h\" (UID: \"e792ccde-27e1-4543-8252-0c5468bab8dc\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h"
Sep 14 07:55:30.297313 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.297175    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e792ccde-27e1-4543-8252-0c5468bab8dc-tls-certs\") pod \"kube-auth-proxy-75fb8dbff8-bfs7h\" (UID: \"e792ccde-27e1-4543-8252-0c5468bab8dc\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h"
Sep 14 07:55:30.299381 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.299354    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e792ccde-27e1-4543-8252-0c5468bab8dc-tmp\") pod \"kube-auth-proxy-75fb8dbff8-bfs7h\" (UID: \"e792ccde-27e1-4543-8252-0c5468bab8dc\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h"
Sep 14 07:55:30.299605 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.299588    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e792ccde-27e1-4543-8252-0c5468bab8dc-tls-certs\") pod \"kube-auth-proxy-75fb8dbff8-bfs7h\" (UID: \"e792ccde-27e1-4543-8252-0c5468bab8dc\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h"
Sep 14 07:55:30.304479 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.304461    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l546f\" (UniqueName: \"kubernetes.io/projected/e792ccde-27e1-4543-8252-0c5468bab8dc-kube-api-access-l546f\") pod \"kube-auth-proxy-75fb8dbff8-bfs7h\" (UID: \"e792ccde-27e1-4543-8252-0c5468bab8dc\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h"
Sep 14 07:55:30.366107 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.366089    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h"
Sep 14 07:55:30.639218 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:30.639185    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h"]
Sep 14 07:55:30.641512 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:55:30.641488    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode792ccde_27e1_4543_8252_0c5468bab8dc.slice/crio-e0ed849c2a0daf07c8436d074c09c9a82990cdb348ec6308639b82dff8b726f5 WatchSource:0}: Error finding container e0ed849c2a0daf07c8436d074c09c9a82990cdb348ec6308639b82dff8b726f5: Status 404 returned error can't find the container with id e0ed849c2a0daf07c8436d074c09c9a82990cdb348ec6308639b82dff8b726f5
Sep 14 07:55:31.202628 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.202598    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"]
Sep 14 07:55:31.207078 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.207056    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.207594 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.207568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:31.209322 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.209304    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 14 07:55:31.209699 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.209682    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6b2sn\""
Sep 14 07:55:31.209853 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.209837    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 14 07:55:31.209922 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.209853    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 07:55:31.209922 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.209873    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 14 07:55:31.222129 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.222107    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"]
Sep 14 07:55:31.272927 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.272887    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:31.303810 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.303781    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.303947 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.303839    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.303947 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.303870    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/328af05d-25b6-4541-8b6d-db3f31f818a6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.303947 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.303894    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.303947 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.303945    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.304159 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.303993    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.304159 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.304017    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6mrf\" (UniqueName: \"kubernetes.io/projected/328af05d-25b6-4541-8b6d-db3f31f818a6-kube-api-access-x6mrf\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.394842 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.394802    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h" event={"ID":"e792ccde-27e1-4543-8252-0c5468bab8dc","Type":"ContainerStarted","Data":"e0ed849c2a0daf07c8436d074c09c9a82990cdb348ec6308639b82dff8b726f5"}
Sep 14 07:55:31.405442 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.405415    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.405571 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.405453    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/328af05d-25b6-4541-8b6d-db3f31f818a6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.405571 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.405475    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.405571 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.405528    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.405728 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.405575    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.405728 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.405598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6mrf\" (UniqueName: \"kubernetes.io/projected/328af05d-25b6-4541-8b6d-db3f31f818a6-kube-api-access-x6mrf\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.405728 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.405635    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.407017 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.406991    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.408800 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.408772    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.409323 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.409295    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.409483 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.409463    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/328af05d-25b6-4541-8b6d-db3f31f818a6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.410479 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.410454    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.413713 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.413690    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6mrf\" (UniqueName: \"kubernetes.io/projected/328af05d-25b6-4541-8b6d-db3f31f818a6-kube-api-access-x6mrf\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.414095 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.414069    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-cf8r4\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.517355 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.517283    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:31.805815 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:31.805777    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"]
Sep 14 07:55:31.808455 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:55:31.808413    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod328af05d_25b6_4541_8b6d_db3f31f818a6.slice/crio-222707884b3e971128cfbde65d37aae79c3e5e07339d4068a2f760daa7ce133a WatchSource:0}: Error finding container 222707884b3e971128cfbde65d37aae79c3e5e07339d4068a2f760daa7ce133a: Status 404 returned error can't find the container with id 222707884b3e971128cfbde65d37aae79c3e5e07339d4068a2f760daa7ce133a
Sep 14 07:55:32.142141 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:32.142101    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:32.403380 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:32.403257    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4" event={"ID":"328af05d-25b6-4541-8b6d-db3f31f818a6","Type":"ContainerStarted","Data":"222707884b3e971128cfbde65d37aae79c3e5e07339d4068a2f760daa7ce133a"}
Sep 14 07:55:34.414553 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:34.414512    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h" event={"ID":"e792ccde-27e1-4543-8252-0c5468bab8dc","Type":"ContainerStarted","Data":"46689d6742dbfafc573c6735ca239804a379b5746b4a689ca5f1f6bba7f3329f"}
Sep 14 07:55:34.431716 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:34.431667    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-bfs7h" podStartSLOduration=1.696940034 podStartE2EDuration="4.43165059s" totalImagesPullingTime="2.734710556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:34.429373003 +0000 UTC m=+389.133911508" watchObservedRunningTime="2026-09-14 07:55:34.43165059 +0000 UTC m=+389.136189093"
Sep 14 07:55:35.142374 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:35.142334    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 07:55:35.142503 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:35.142418    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 07:55:35.419938 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:35.419861    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4" event={"ID":"328af05d-25b6-4541-8b6d-db3f31f818a6","Type":"ContainerStarted","Data":"e3ce0543e84e88d37ef1b92039bbec9613a6a53155519b149bba689520604193"}
Sep 14 07:55:35.419938 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:35.419926    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:35.446947 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:35.446896    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4" podStartSLOduration=1.115813306 podStartE2EDuration="4.446882845s" totalImagesPullingTime="3.331069539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:35.446496222 +0000 UTC m=+390.151034725" watchObservedRunningTime="2026-09-14 07:55:35.446882845 +0000 UTC m=+390.151421347"
Sep 14 07:55:35.779103 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:35.779012    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:35.880329 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:35.880302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:35.981104 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:35.981077    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:36.080834 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.080748    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:36.160235 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.160179    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:36.425981 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.425951    2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-cf8r4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 07:55:36.426360 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.426008    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4" podUID="328af05d-25b6-4541-8b6d-db3f31f818a6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 07:55:36.633345 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.633313    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"]
Sep 14 07:55:36.637197 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.637174    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.639622 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.639584    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-j99j8\""
Sep 14 07:55:36.648703 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.648677    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"]
Sep 14 07:55:36.657967 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.657946    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:36.750785 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.750724    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/930c3440-6e72-4f25-baf8-5fca805f538d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.750785 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.750761    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.750785 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.750778    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/930c3440-6e72-4f25-baf8-5fca805f538d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.750951 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.750808    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.750951 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.750834    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/930c3440-6e72-4f25-baf8-5fca805f538d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.750951 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.750911    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.751055 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.750955    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.751055 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.750994    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2p9c\" (UniqueName: \"kubernetes.io/projected/930c3440-6e72-4f25-baf8-5fca805f538d-kube-api-access-r2p9c\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.751055 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.751045    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.846422 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.846393    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:36.852168 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.852142    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.852270 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.852175    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/930c3440-6e72-4f25-baf8-5fca805f538d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.852270 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.852200    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.852270 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.852244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/930c3440-6e72-4f25-baf8-5fca805f538d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.852444 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.852272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.852444 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.852291    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.852444 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.852320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2p9c\" (UniqueName: \"kubernetes.io/projected/930c3440-6e72-4f25-baf8-5fca805f538d-kube-api-access-r2p9c\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.852444 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.852360    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.852444 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.852392    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/930c3440-6e72-4f25-baf8-5fca805f538d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.852835 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.852788    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.852926 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.852866    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.852926 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.852875    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.853000 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.852939    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.853091 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.853075    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/930c3440-6e72-4f25-baf8-5fca805f538d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.854727 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.854708    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/930c3440-6e72-4f25-baf8-5fca805f538d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.854867 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.854849    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/930c3440-6e72-4f25-baf8-5fca805f538d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.862303 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.862280    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/930c3440-6e72-4f25-baf8-5fca805f538d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.862942 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.862922    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2p9c\" (UniqueName: \"kubernetes.io/projected/930c3440-6e72-4f25-baf8-5fca805f538d-kube-api-access-r2p9c\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp\" (UID: \"930c3440-6e72-4f25-baf8-5fca805f538d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:36.946637 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:36.946617    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:37.228003 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:37.227976    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"]
Sep 14 07:55:37.230476 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:37.230441    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:37.230706 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:55:37.230667    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod930c3440_6e72_4f25_baf8_5fca805f538d.slice/crio-c146567fa924dea3a97dfa6ed3bd999a69c102771de16f10e2d7c3bba209749f WatchSource:0}: Error finding container c146567fa924dea3a97dfa6ed3bd999a69c102771de16f10e2d7c3bba209749f: Status 404 returned error can't find the container with id c146567fa924dea3a97dfa6ed3bd999a69c102771de16f10e2d7c3bba209749f
Sep 14 07:55:37.430254 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:37.430200    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp" event={"ID":"930c3440-6e72-4f25-baf8-5fca805f538d","Type":"ContainerStarted","Data":"c146567fa924dea3a97dfa6ed3bd999a69c102771de16f10e2d7c3bba209749f"}
Sep 14 07:55:39.425365 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:39.425329    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:40.191321 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:40.191273    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 07:55:40.191421 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:40.191359    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 07:55:40.191421 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:40.191388    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 07:55:40.426184 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:40.426130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:40.447453 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:40.447378    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp" event={"ID":"930c3440-6e72-4f25-baf8-5fca805f538d","Type":"ContainerStarted","Data":"f8d38120fda52e6d077db764f59c9c55fa569b04b00f6243ff5ff6c889ba58bd"}
Sep 14 07:55:40.523980 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:40.523952    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:40.718986 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:40.718913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:40.869589 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:40.869552    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:40.947617 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:40.947571    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:41.027567 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:41.027484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:41.952164 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:41.952132    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:41.974650 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:41.974588    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp" podStartSLOduration=3.0162868 podStartE2EDuration="5.974565766s" totalImagesPullingTime="2.958278966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:40.471342259 +0000 UTC m=+395.175880801" watchObservedRunningTime="2026-09-14 07:55:41.974565766 +0000 UTC m=+396.679104269"
Sep 14 07:55:42.457242 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:42.457196    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:42.458172 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:42.458151    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp"
Sep 14 07:55:47.395249 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.395187    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"]
Sep 14 07:55:47.403042 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.403008    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.424775 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.424748    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"]
Sep 14 07:55:47.436885 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.436835    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.437025 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.436914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.437025 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.436995    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.437136 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.437035    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwxbq\" (UniqueName: \"kubernetes.io/projected/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-kube-api-access-mwxbq\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.437136 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.437066    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.437136 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.437124    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.437307 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.437159    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.537632 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.537602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.537799 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.537645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwxbq\" (UniqueName: \"kubernetes.io/projected/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-kube-api-access-mwxbq\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.537799 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.537765    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.537885 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.537821    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.537885 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.537856    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.537976 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.537912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.537976 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.537943    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.538430 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.538402    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.540266 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.540232    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.540372 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.540356    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.540713 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.540693    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.540757 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.540704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.545074 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.545046    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.545196 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.545181    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwxbq\" (UniqueName: \"kubernetes.io/projected/9f4d1bab-2cfb-4cba-995b-16f0463b3c37-kube-api-access-mwxbq\") pod \"istiod-openshift-gateway-5677ccb49b-hhp85\" (UID: \"9f4d1bab-2cfb-4cba-995b-16f0463b3c37\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:47.714283 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:47.714193    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:48.015200 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:48.015131    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"]
Sep 14 07:55:48.018648 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:55:48.018620    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f4d1bab_2cfb_4cba_995b_16f0463b3c37.slice/crio-fa885e9b02da745d7ccc13f3ab47766d87ae3a9adc0b697f95dac0af14b647b5 WatchSource:0}: Error finding container fa885e9b02da745d7ccc13f3ab47766d87ae3a9adc0b697f95dac0af14b647b5: Status 404 returned error can't find the container with id fa885e9b02da745d7ccc13f3ab47766d87ae3a9adc0b697f95dac0af14b647b5
Sep 14 07:55:48.020784 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:48.020746    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 07:55:48.020846 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:48.020815    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 07:55:48.487803 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:48.487762    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85" event={"ID":"9f4d1bab-2cfb-4cba-995b-16f0463b3c37","Type":"ContainerStarted","Data":"b073a32527cd4eb1c8466eedd1aa975375b0a77fcf44236ead9d52a2dd825f53"}
Sep 14 07:55:48.488269 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:48.487824    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85" event={"ID":"9f4d1bab-2cfb-4cba-995b-16f0463b3c37","Type":"ContainerStarted","Data":"fa885e9b02da745d7ccc13f3ab47766d87ae3a9adc0b697f95dac0af14b647b5"}
Sep 14 07:55:48.488269 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:48.487890    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:48.523403 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:48.523354    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85" podStartSLOduration=1.523338614 podStartE2EDuration="1.523338614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:48.521039034 +0000 UTC m=+403.225577563" watchObservedRunningTime="2026-09-14 07:55:48.523338614 +0000 UTC m=+403.227877116"
Sep 14 07:55:49.493983 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.493954    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hhp85"
Sep 14 07:55:49.592101 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.592062    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"]
Sep 14 07:55:49.592346 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.592320    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4" podUID="328af05d-25b6-4541-8b6d-db3f31f818a6" containerName="discovery" containerID="cri-o://e3ce0543e84e88d37ef1b92039bbec9613a6a53155519b149bba689520604193" gracePeriod=30
Sep 14 07:55:49.828509 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.828485    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:49.857901 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.857858    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-csr-dns-cert\") pod \"328af05d-25b6-4541-8b6d-db3f31f818a6\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") "
Sep 14 07:55:49.858093 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.857905    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6mrf\" (UniqueName: \"kubernetes.io/projected/328af05d-25b6-4541-8b6d-db3f31f818a6-kube-api-access-x6mrf\") pod \"328af05d-25b6-4541-8b6d-db3f31f818a6\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") "
Sep 14 07:55:49.858093 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.857943    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/328af05d-25b6-4541-8b6d-db3f31f818a6-local-certs\") pod \"328af05d-25b6-4541-8b6d-db3f31f818a6\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") "
Sep 14 07:55:49.858093 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.857992    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-cacerts\") pod \"328af05d-25b6-4541-8b6d-db3f31f818a6\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") "
Sep 14 07:55:49.858093 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.858018    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-token\") pod \"328af05d-25b6-4541-8b6d-db3f31f818a6\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") "
Sep 14 07:55:49.858093 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.858075    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-kubeconfig\") pod \"328af05d-25b6-4541-8b6d-db3f31f818a6\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") "
Sep 14 07:55:49.858364 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.858114    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-csr-ca-configmap\") pod \"328af05d-25b6-4541-8b6d-db3f31f818a6\" (UID: \"328af05d-25b6-4541-8b6d-db3f31f818a6\") "
Sep 14 07:55:49.858729 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.858684    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "328af05d-25b6-4541-8b6d-db3f31f818a6" (UID: "328af05d-25b6-4541-8b6d-db3f31f818a6"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:55:49.861057 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.861030    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "328af05d-25b6-4541-8b6d-db3f31f818a6" (UID: "328af05d-25b6-4541-8b6d-db3f31f818a6"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:55:49.861572 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.861546    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-cacerts" (OuterVolumeSpecName: "cacerts") pod "328af05d-25b6-4541-8b6d-db3f31f818a6" (UID: "328af05d-25b6-4541-8b6d-db3f31f818a6"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:55:49.861681 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.861554    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "328af05d-25b6-4541-8b6d-db3f31f818a6" (UID: "328af05d-25b6-4541-8b6d-db3f31f818a6"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:55:49.861811 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.861793    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/328af05d-25b6-4541-8b6d-db3f31f818a6-local-certs" (OuterVolumeSpecName: "local-certs") pod "328af05d-25b6-4541-8b6d-db3f31f818a6" (UID: "328af05d-25b6-4541-8b6d-db3f31f818a6"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:55:49.862051 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.862024    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/328af05d-25b6-4541-8b6d-db3f31f818a6-kube-api-access-x6mrf" (OuterVolumeSpecName: "kube-api-access-x6mrf") pod "328af05d-25b6-4541-8b6d-db3f31f818a6" (UID: "328af05d-25b6-4541-8b6d-db3f31f818a6"). InnerVolumeSpecName "kube-api-access-x6mrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:55:49.862132 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.862092    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "328af05d-25b6-4541-8b6d-db3f31f818a6" (UID: "328af05d-25b6-4541-8b6d-db3f31f818a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:55:49.881133 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.881099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:49.939059 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.939028    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8"]
Sep 14 07:55:49.939456 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.939438    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="328af05d-25b6-4541-8b6d-db3f31f818a6" containerName="discovery"
Sep 14 07:55:49.939513 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.939460    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="328af05d-25b6-4541-8b6d-db3f31f818a6" containerName="discovery"
Sep 14 07:55:49.939555 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.939543    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="328af05d-25b6-4541-8b6d-db3f31f818a6" containerName="discovery"
Sep 14 07:55:49.941867 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.941852    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8"
Sep 14 07:55:49.944536 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.944519    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 14 07:55:49.944646 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.944566    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-rztbk\""
Sep 14 07:55:49.952814 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.952787    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8"]
Sep 14 07:55:49.958713 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.958695    2576 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-cacerts\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 07:55:49.958713 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.958714    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 07:55:49.958825 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.958723    2576 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-kubeconfig\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 07:55:49.958825 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.958732    2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-csr-ca-configmap\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 07:55:49.958825 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.958741    2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/328af05d-25b6-4541-8b6d-db3f31f818a6-istio-csr-dns-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 07:55:49.958825 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.958750    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6mrf\" (UniqueName: \"kubernetes.io/projected/328af05d-25b6-4541-8b6d-db3f31f818a6-kube-api-access-x6mrf\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 07:55:49.958825 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.958759    2576 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/328af05d-25b6-4541-8b6d-db3f31f818a6-local-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 07:55:49.971378 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:49.971347    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:50.059039 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.058978    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2b42013-7304-48c6-9016-5b64df1a32b4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tqvk8\" (UID: \"f2b42013-7304-48c6-9016-5b64df1a32b4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8"
Sep 14 07:55:50.059039 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.059034    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx9f5\" (UniqueName: \"kubernetes.io/projected/f2b42013-7304-48c6-9016-5b64df1a32b4-kube-api-access-lx9f5\") pod \"llmisvc-controller-manager-777c6bd7d5-tqvk8\" (UID: \"f2b42013-7304-48c6-9016-5b64df1a32b4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8"
Sep 14 07:55:50.160013 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.159979    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2b42013-7304-48c6-9016-5b64df1a32b4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tqvk8\" (UID: \"f2b42013-7304-48c6-9016-5b64df1a32b4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8"
Sep 14 07:55:50.160150 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.160029    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx9f5\" (UniqueName: \"kubernetes.io/projected/f2b42013-7304-48c6-9016-5b64df1a32b4-kube-api-access-lx9f5\") pod \"llmisvc-controller-manager-777c6bd7d5-tqvk8\" (UID: \"f2b42013-7304-48c6-9016-5b64df1a32b4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8"
Sep 14 07:55:50.160230 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:55:50.160145    2576 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 14 07:55:50.160294 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:55:50.160244    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2b42013-7304-48c6-9016-5b64df1a32b4-cert podName:f2b42013-7304-48c6-9016-5b64df1a32b4 nodeName:}" failed. No retries permitted until 2026-09-14 07:55:50.660222939 +0000 UTC m=+405.364761424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f2b42013-7304-48c6-9016-5b64df1a32b4-cert") pod "llmisvc-controller-manager-777c6bd7d5-tqvk8" (UID: "f2b42013-7304-48c6-9016-5b64df1a32b4") : secret "llmisvc-webhook-server-cert" not found
Sep 14 07:55:50.176902 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.176879    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx9f5\" (UniqueName: \"kubernetes.io/projected/f2b42013-7304-48c6-9016-5b64df1a32b4-kube-api-access-lx9f5\") pod \"llmisvc-controller-manager-777c6bd7d5-tqvk8\" (UID: \"f2b42013-7304-48c6-9016-5b64df1a32b4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8"
Sep 14 07:55:50.333826 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.333667    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:50.432844 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.432805    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:50.497732 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.497696    2576 generic.go:356] "Generic (PLEG): container finished" podID="328af05d-25b6-4541-8b6d-db3f31f818a6" containerID="e3ce0543e84e88d37ef1b92039bbec9613a6a53155519b149bba689520604193" exitCode=0
Sep 14 07:55:50.498131 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.497744    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"
Sep 14 07:55:50.498131 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.497784    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4" event={"ID":"328af05d-25b6-4541-8b6d-db3f31f818a6","Type":"ContainerDied","Data":"e3ce0543e84e88d37ef1b92039bbec9613a6a53155519b149bba689520604193"}
Sep 14 07:55:50.498131 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.498113    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4" event={"ID":"328af05d-25b6-4541-8b6d-db3f31f818a6","Type":"ContainerDied","Data":"222707884b3e971128cfbde65d37aae79c3e5e07339d4068a2f760daa7ce133a"}
Sep 14 07:55:50.498326 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.498185    2576 scope.go:117] "RemoveContainer" containerID="e3ce0543e84e88d37ef1b92039bbec9613a6a53155519b149bba689520604193"
Sep 14 07:55:50.511995 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.511973    2576 scope.go:117] "RemoveContainer" containerID="e3ce0543e84e88d37ef1b92039bbec9613a6a53155519b149bba689520604193"
Sep 14 07:55:50.512291 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:55:50.512271    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3ce0543e84e88d37ef1b92039bbec9613a6a53155519b149bba689520604193\": container with ID starting with e3ce0543e84e88d37ef1b92039bbec9613a6a53155519b149bba689520604193 not found: ID does not exist" containerID="e3ce0543e84e88d37ef1b92039bbec9613a6a53155519b149bba689520604193"
Sep 14 07:55:50.512357 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.512299    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3ce0543e84e88d37ef1b92039bbec9613a6a53155519b149bba689520604193"} err="failed to get container status \"e3ce0543e84e88d37ef1b92039bbec9613a6a53155519b149bba689520604193\": rpc error: code = NotFound desc = could not find container \"e3ce0543e84e88d37ef1b92039bbec9613a6a53155519b149bba689520604193\": container with ID starting with e3ce0543e84e88d37ef1b92039bbec9613a6a53155519b149bba689520604193 not found: ID does not exist"
Sep 14 07:55:50.521845 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.521825    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"]
Sep 14 07:55:50.526084 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.526063    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cf8r4"]
Sep 14 07:55:50.663542 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.663518    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2b42013-7304-48c6-9016-5b64df1a32b4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tqvk8\" (UID: \"f2b42013-7304-48c6-9016-5b64df1a32b4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8"
Sep 14 07:55:50.665769 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.665745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2b42013-7304-48c6-9016-5b64df1a32b4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tqvk8\" (UID: \"f2b42013-7304-48c6-9016-5b64df1a32b4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8"
Sep 14 07:55:50.855639 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:50.855620    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8"
Sep 14 07:55:51.127236 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:51.127190    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8"]
Sep 14 07:55:51.128851 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:55:51.128814    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf2b42013_7304_48c6_9016_5b64df1a32b4.slice/crio-3675d3fa92a3cbf556083c6df0515c8302bf8fd5bf7fc7a3b67e2e58b2416dab WatchSource:0}: Error finding container 3675d3fa92a3cbf556083c6df0515c8302bf8fd5bf7fc7a3b67e2e58b2416dab: Status 404 returned error can't find the container with id 3675d3fa92a3cbf556083c6df0515c8302bf8fd5bf7fc7a3b67e2e58b2416dab
Sep 14 07:55:51.153381 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:51.153358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:51.294393 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:51.294370    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:51.362147 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:51.362116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:51.502802 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:51.502739    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8" event={"ID":"f2b42013-7304-48c6-9016-5b64df1a32b4","Type":"ContainerStarted","Data":"3675d3fa92a3cbf556083c6df0515c8302bf8fd5bf7fc7a3b67e2e58b2416dab"}
Sep 14 07:55:51.839818 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:51.839748    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="328af05d-25b6-4541-8b6d-db3f31f818a6" path="/var/lib/kubelet/pods/328af05d-25b6-4541-8b6d-db3f31f818a6/volumes"
Sep 14 07:55:54.520998 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:54.520966    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8" event={"ID":"f2b42013-7304-48c6-9016-5b64df1a32b4","Type":"ContainerStarted","Data":"aa9ed9768a757e10de879aa48845615518b7b9da82c74a2c0ac056724488e9e7"}
Sep 14 07:55:54.521405 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:54.521081    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8"
Sep 14 07:55:54.559830 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:55:54.559777    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8" podStartSLOduration=2.8761066939999997 podStartE2EDuration="5.559760363s" totalImagesPullingTime="2.683653669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:54.558942522 +0000 UTC m=+409.263481024" watchObservedRunningTime="2026-09-14 07:55:54.559760363 +0000 UTC m=+409.264298866"
Sep 14 07:56:25.527551 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:56:25.527523    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tqvk8"
Sep 14 07:56:41.617011 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:56:41.616928    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:41.757087 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:56:41.757058    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:00.846136 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:00.846083    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:01.266141 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:01.266111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:04.069519 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:04.069482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:05.329098 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:05.329061    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7rgrt"]
Sep 14 07:57:05.332371 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:05.332348    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7rgrt"
Sep 14 07:57:05.336437 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:05.336411    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 14 07:57:05.336578 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:05.336436    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 14 07:57:05.336719 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:05.336700    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-n5sz4\""
Sep 14 07:57:05.336719 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:05.336714    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 14 07:57:05.351607 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:05.351582    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7rgrt"]
Sep 14 07:57:05.417258 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:05.417227    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bwmw\" (UniqueName: \"kubernetes.io/projected/f8de7c18-9ea3-4fb9-a893-6d328dc4257a-kube-api-access-8bwmw\") pod \"dns-operator-controller-manager-7b84cd7c9c-7rgrt\" (UID: \"f8de7c18-9ea3-4fb9-a893-6d328dc4257a\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7rgrt"
Sep 14 07:57:05.518543 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:05.518513    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bwmw\" (UniqueName: \"kubernetes.io/projected/f8de7c18-9ea3-4fb9-a893-6d328dc4257a-kube-api-access-8bwmw\") pod \"dns-operator-controller-manager-7b84cd7c9c-7rgrt\" (UID: \"f8de7c18-9ea3-4fb9-a893-6d328dc4257a\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7rgrt"
Sep 14 07:57:05.530312 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:05.530290    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bwmw\" (UniqueName: \"kubernetes.io/projected/f8de7c18-9ea3-4fb9-a893-6d328dc4257a-kube-api-access-8bwmw\") pod \"dns-operator-controller-manager-7b84cd7c9c-7rgrt\" (UID: \"f8de7c18-9ea3-4fb9-a893-6d328dc4257a\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7rgrt"
Sep 14 07:57:05.643815 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:05.643791    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7rgrt"
Sep 14 07:57:05.656220 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:05.656184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:05.884423 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:05.884391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:05.912960 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:05.912902    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7rgrt"]
Sep 14 07:57:05.915091 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:57:05.915069    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8de7c18_9ea3_4fb9_a893_6d328dc4257a.slice/crio-7343e3c625146b731d7c4cc06d2bb9633f097a6e037f412b17fc6ace8b44b86f WatchSource:0}: Error finding container 7343e3c625146b731d7c4cc06d2bb9633f097a6e037f412b17fc6ace8b44b86f: Status 404 returned error can't find the container with id 7343e3c625146b731d7c4cc06d2bb9633f097a6e037f412b17fc6ace8b44b86f
Sep 14 07:57:06.059338 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:06.059314    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:06.163543 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:06.163481    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:06.846146 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:06.846068    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7rgrt" event={"ID":"f8de7c18-9ea3-4fb9-a893-6d328dc4257a","Type":"ContainerStarted","Data":"7343e3c625146b731d7c4cc06d2bb9633f097a6e037f412b17fc6ace8b44b86f"}
Sep 14 07:57:07.309168 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:07.309134    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:08.263083 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.253751    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-q2qzk"]
Sep 14 07:57:08.265598 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.265569    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-q2qzk"
Sep 14 07:57:08.268483 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.268266    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-l2fqs\""
Sep 14 07:57:08.281064 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.280996    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-q2qzk"]
Sep 14 07:57:08.338503 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.338471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldbr2\" (UniqueName: \"kubernetes.io/projected/25e8a50a-1b49-4810-916e-d1e7e06ca1be-kube-api-access-ldbr2\") pod \"authorino-operator-76b478bd78-q2qzk\" (UID: \"25e8a50a-1b49-4810-916e-d1e7e06ca1be\") " pod="kuadrant-system/authorino-operator-76b478bd78-q2qzk"
Sep 14 07:57:08.439532 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.439496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldbr2\" (UniqueName: \"kubernetes.io/projected/25e8a50a-1b49-4810-916e-d1e7e06ca1be-kube-api-access-ldbr2\") pod \"authorino-operator-76b478bd78-q2qzk\" (UID: \"25e8a50a-1b49-4810-916e-d1e7e06ca1be\") " pod="kuadrant-system/authorino-operator-76b478bd78-q2qzk"
Sep 14 07:57:08.457921 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.457891    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldbr2\" (UniqueName: \"kubernetes.io/projected/25e8a50a-1b49-4810-916e-d1e7e06ca1be-kube-api-access-ldbr2\") pod \"authorino-operator-76b478bd78-q2qzk\" (UID: \"25e8a50a-1b49-4810-916e-d1e7e06ca1be\") " pod="kuadrant-system/authorino-operator-76b478bd78-q2qzk"
Sep 14 07:57:08.512519 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.512487    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:08.583166 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.583097    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-q2qzk"
Sep 14 07:57:08.646126 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.646103    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 14 07:57:08.653778 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.653748    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:08.857022 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.856996    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7rgrt" event={"ID":"f8de7c18-9ea3-4fb9-a893-6d328dc4257a","Type":"ContainerStarted","Data":"268ce39a2796ecd4a0fef0fcc7c8423cb42a8ef0b20a1c56c5a942720eeddeaf"}
Sep 14 07:57:08.857171 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.857128    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7rgrt"
Sep 14 07:57:08.883591 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.883569    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-q2qzk"]
Sep 14 07:57:08.886083 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:57:08.886049    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25e8a50a_1b49_4810_916e_d1e7e06ca1be.slice/crio-735707b3ef5c77b0decb8e7a12cd7fa1ad89f00eeb33aa68d41caad6099b5af2 WatchSource:0}: Error finding container 735707b3ef5c77b0decb8e7a12cd7fa1ad89f00eeb33aa68d41caad6099b5af2: Status 404 returned error can't find the container with id 735707b3ef5c77b0decb8e7a12cd7fa1ad89f00eeb33aa68d41caad6099b5af2
Sep 14 07:57:08.890410 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:08.890368    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7rgrt" podStartSLOduration=1.165778331 podStartE2EDuration="3.890353126s" totalImagesPullingTime="2.724574795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:57:08.888857825 +0000 UTC m=+483.593396338" watchObservedRunningTime="2026-09-14 07:57:08.890353126 +0000 UTC m=+483.594891627"
Sep 14 07:57:09.246967 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:09.246886    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:09.513308 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:09.513238    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:09.653513 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:09.653479    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:09.757936 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:09.757899    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:09.863613 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:09.863578    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-q2qzk" event={"ID":"25e8a50a-1b49-4810-916e-d1e7e06ca1be","Type":"ContainerStarted","Data":"735707b3ef5c77b0decb8e7a12cd7fa1ad89f00eeb33aa68d41caad6099b5af2"}
Sep 14 07:57:10.869465 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:10.869430    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-q2qzk" event={"ID":"25e8a50a-1b49-4810-916e-d1e7e06ca1be","Type":"ContainerStarted","Data":"47ee732afcd31d9c5777ca9fb90638c1384daed97a08607160bd867e30106e78"}
Sep 14 07:57:10.869829 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:10.869537    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-q2qzk"
Sep 14 07:57:10.890124 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:10.890080    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-q2qzk" podStartSLOduration=1.5277922510000002 podStartE2EDuration="2.890068165s" totalImagesPullingTime="1.362275914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:57:10.888351029 +0000 UTC m=+485.592889532" watchObservedRunningTime="2026-09-14 07:57:10.890068165 +0000 UTC m=+485.594606666"
Sep 14 07:57:15.267544 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:15.267514    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:15.367537 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:15.367499    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:18.274031 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:18.273991    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:18.374880 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:18.374826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:19.611284 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:19.611253    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:19.866097 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:19.866025    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7rgrt"
Sep 14 07:57:19.929976 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:19.929947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:20.028793 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:20.028765    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:20.170225 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:20.170121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:21.875827 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:21.875793    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-q2qzk"
Sep 14 07:57:24.059551 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:24.059476    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:24.159104 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:24.159073    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:24.656078 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:24.656023    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:24.756497 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:24.756461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:29.250439 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:29.250402    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"]
Sep 14 07:57:29.253512 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:29.253495    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"
Sep 14 07:57:29.255800 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:29.255775    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vb5sh\""
Sep 14 07:57:29.267337 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:29.267313    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"]
Sep 14 07:57:29.393829 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:29.393800    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv2tl\" (UniqueName: \"kubernetes.io/projected/031fbc82-0bc0-47ed-9ed1-984dcefe3d64-kube-api-access-bv2tl\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-88kbs\" (UID: \"031fbc82-0bc0-47ed-9ed1-984dcefe3d64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"
Sep 14 07:57:29.393939 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:29.393843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/031fbc82-0bc0-47ed-9ed1-984dcefe3d64-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-88kbs\" (UID: \"031fbc82-0bc0-47ed-9ed1-984dcefe3d64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"
Sep 14 07:57:29.494672 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:29.494642    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv2tl\" (UniqueName: \"kubernetes.io/projected/031fbc82-0bc0-47ed-9ed1-984dcefe3d64-kube-api-access-bv2tl\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-88kbs\" (UID: \"031fbc82-0bc0-47ed-9ed1-984dcefe3d64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"
Sep 14 07:57:29.494791 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:29.494681    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/031fbc82-0bc0-47ed-9ed1-984dcefe3d64-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-88kbs\" (UID: \"031fbc82-0bc0-47ed-9ed1-984dcefe3d64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"
Sep 14 07:57:29.495003 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:29.494986    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/031fbc82-0bc0-47ed-9ed1-984dcefe3d64-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-88kbs\" (UID: \"031fbc82-0bc0-47ed-9ed1-984dcefe3d64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"
Sep 14 07:57:29.506191 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:29.506136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv2tl\" (UniqueName: \"kubernetes.io/projected/031fbc82-0bc0-47ed-9ed1-984dcefe3d64-kube-api-access-bv2tl\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-88kbs\" (UID: \"031fbc82-0bc0-47ed-9ed1-984dcefe3d64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"
Sep 14 07:57:29.563280 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:29.563257    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"
Sep 14 07:57:29.842422 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:29.842397    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"]
Sep 14 07:57:29.844069 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:57:29.844042    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod031fbc82_0bc0_47ed_9ed1_984dcefe3d64.slice/crio-7eb801fdd70549613fddca58c57e4b842dbdf319d27134433ae7aa48fc5671ba WatchSource:0}: Error finding container 7eb801fdd70549613fddca58c57e4b842dbdf319d27134433ae7aa48fc5671ba: Status 404 returned error can't find the container with id 7eb801fdd70549613fddca58c57e4b842dbdf319d27134433ae7aa48fc5671ba
Sep 14 07:57:29.951435 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:29.951405    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs" event={"ID":"031fbc82-0bc0-47ed-9ed1-984dcefe3d64","Type":"ContainerStarted","Data":"7eb801fdd70549613fddca58c57e4b842dbdf319d27134433ae7aa48fc5671ba"}
Sep 14 07:57:30.106803 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.106778    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:30.215226 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.215182    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:30.459118 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.459056    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"]
Sep 14 07:57:30.475061 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.475040    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"]
Sep 14 07:57:30.511307 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.511280    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"]
Sep 14 07:57:30.521721 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.521701    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"
Sep 14 07:57:30.534874 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.534846    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"]
Sep 14 07:57:30.605032 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.605001    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq6p6\" (UniqueName: \"kubernetes.io/projected/f6ad5b8b-fe35-46bd-8f67-64a946091b41-kube-api-access-zq6p6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t2td8\" (UID: \"f6ad5b8b-fe35-46bd-8f67-64a946091b41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"
Sep 14 07:57:30.605148 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.605089    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6ad5b8b-fe35-46bd-8f67-64a946091b41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t2td8\" (UID: \"f6ad5b8b-fe35-46bd-8f67-64a946091b41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"
Sep 14 07:57:30.671162 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.671130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:30.706110 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.706089    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6ad5b8b-fe35-46bd-8f67-64a946091b41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t2td8\" (UID: \"f6ad5b8b-fe35-46bd-8f67-64a946091b41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"
Sep 14 07:57:30.706188 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.706122    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq6p6\" (UniqueName: \"kubernetes.io/projected/f6ad5b8b-fe35-46bd-8f67-64a946091b41-kube-api-access-zq6p6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t2td8\" (UID: \"f6ad5b8b-fe35-46bd-8f67-64a946091b41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"
Sep 14 07:57:30.706470 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.706452    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6ad5b8b-fe35-46bd-8f67-64a946091b41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t2td8\" (UID: \"f6ad5b8b-fe35-46bd-8f67-64a946091b41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"
Sep 14 07:57:30.714274 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.714223    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq6p6\" (UniqueName: \"kubernetes.io/projected/f6ad5b8b-fe35-46bd-8f67-64a946091b41-kube-api-access-zq6p6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t2td8\" (UID: \"f6ad5b8b-fe35-46bd-8f67-64a946091b41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"
Sep 14 07:57:30.775138 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.775113    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:30.835901 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:30.835879    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"
Sep 14 07:57:31.124572 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:31.124543    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"]
Sep 14 07:57:31.127891 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:57:31.127863    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6ad5b8b_fe35_46bd_8f67_64a946091b41.slice/crio-517750ebc07899cf69c3fdc706270e7e7b089691aa9d09cc5051a4f9234906dc WatchSource:0}: Error finding container 517750ebc07899cf69c3fdc706270e7e7b089691aa9d09cc5051a4f9234906dc: Status 404 returned error can't find the container with id 517750ebc07899cf69c3fdc706270e7e7b089691aa9d09cc5051a4f9234906dc
Sep 14 07:57:31.279923 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:31.279888    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:31.382157 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:31.382099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:31.964091 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:31.964058    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8" event={"ID":"f6ad5b8b-fe35-46bd-8f67-64a946091b41","Type":"ContainerStarted","Data":"517750ebc07899cf69c3fdc706270e7e7b089691aa9d09cc5051a4f9234906dc"}
Sep 14 07:57:34.982571 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:34.982531    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8" event={"ID":"f6ad5b8b-fe35-46bd-8f67-64a946091b41","Type":"ContainerStarted","Data":"05fdbb82d5f9b5a333b087f5f0686e83df2e2a7f6954425ff22c3a16584c720e"}
Sep 14 07:57:34.982976 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:34.982601    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"
Sep 14 07:57:34.984385 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:34.984359    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs" podUID="031fbc82-0bc0-47ed-9ed1-984dcefe3d64" containerName="manager" containerID="cri-o://feabcac67dfc4d89b829f545f1407cc4285719eebbb5eb46f834debe034bf07a" gracePeriod=2
Sep 14 07:57:35.004197 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:35.004113    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8" podStartSLOduration=1.294672494 podStartE2EDuration="5.004096415s" totalImagesPullingTime="3.709423921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:57:35.002605424 +0000 UTC m=+509.707143925" watchObservedRunningTime="2026-09-14 07:57:35.004096415 +0000 UTC m=+509.708634918"
Sep 14 07:57:35.193587 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:35.193562    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"
Sep 14 07:57:35.195826 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:57:35.195795    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-88kbs\" is forbidden: User \"system:node:ip-10-0-133-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-123.ec2.internal' and this object" podUID="031fbc82-0bc0-47ed-9ed1-984dcefe3d64" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"
Sep 14 07:57:35.342931 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:35.342903    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bv2tl\" (UniqueName: \"kubernetes.io/projected/031fbc82-0bc0-47ed-9ed1-984dcefe3d64-kube-api-access-bv2tl\") pod \"031fbc82-0bc0-47ed-9ed1-984dcefe3d64\" (UID: \"031fbc82-0bc0-47ed-9ed1-984dcefe3d64\") "
Sep 14 07:57:35.343058 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:35.342959    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/031fbc82-0bc0-47ed-9ed1-984dcefe3d64-extensions-socket-volume\") pod \"031fbc82-0bc0-47ed-9ed1-984dcefe3d64\" (UID: \"031fbc82-0bc0-47ed-9ed1-984dcefe3d64\") "
Sep 14 07:57:35.343245 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:35.343194    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031fbc82-0bc0-47ed-9ed1-984dcefe3d64-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "031fbc82-0bc0-47ed-9ed1-984dcefe3d64" (UID: "031fbc82-0bc0-47ed-9ed1-984dcefe3d64"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:57:35.345026 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:35.345003    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/031fbc82-0bc0-47ed-9ed1-984dcefe3d64-kube-api-access-bv2tl" (OuterVolumeSpecName: "kube-api-access-bv2tl") pod "031fbc82-0bc0-47ed-9ed1-984dcefe3d64" (UID: "031fbc82-0bc0-47ed-9ed1-984dcefe3d64"). InnerVolumeSpecName "kube-api-access-bv2tl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:57:35.443482 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:35.443457    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/031fbc82-0bc0-47ed-9ed1-984dcefe3d64-extensions-socket-volume\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 07:57:35.443482 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:35.443479    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bv2tl\" (UniqueName: \"kubernetes.io/projected/031fbc82-0bc0-47ed-9ed1-984dcefe3d64-kube-api-access-bv2tl\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 07:57:35.836775 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:57:35.836695    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-88kbs\" is forbidden: User \"system:node:ip-10-0-133-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-123.ec2.internal' and this object" podUID="031fbc82-0bc0-47ed-9ed1-984dcefe3d64" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"
Sep 14 07:57:35.838454 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:35.838431    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="031fbc82-0bc0-47ed-9ed1-984dcefe3d64" path="/var/lib/kubelet/pods/031fbc82-0bc0-47ed-9ed1-984dcefe3d64/volumes"
Sep 14 07:57:35.990761 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:35.990733    2576 generic.go:356] "Generic (PLEG): container finished" podID="031fbc82-0bc0-47ed-9ed1-984dcefe3d64" containerID="feabcac67dfc4d89b829f545f1407cc4285719eebbb5eb46f834debe034bf07a" exitCode=2
Sep 14 07:57:35.991091 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:35.990776    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"
Sep 14 07:57:35.991091 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:35.990820    2576 scope.go:117] "RemoveContainer" containerID="feabcac67dfc4d89b829f545f1407cc4285719eebbb5eb46f834debe034bf07a"
Sep 14 07:57:35.994918 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:57:35.994890    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-88kbs\" is forbidden: User \"system:node:ip-10-0-133-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-123.ec2.internal' and this object" podUID="031fbc82-0bc0-47ed-9ed1-984dcefe3d64" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-88kbs"
Sep 14 07:57:35.999189 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:35.999171    2576 scope.go:117] "RemoveContainer" containerID="feabcac67dfc4d89b829f545f1407cc4285719eebbb5eb46f834debe034bf07a"
Sep 14 07:57:35.999495 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:57:35.999475    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feabcac67dfc4d89b829f545f1407cc4285719eebbb5eb46f834debe034bf07a\": container with ID starting with feabcac67dfc4d89b829f545f1407cc4285719eebbb5eb46f834debe034bf07a not found: ID does not exist" containerID="feabcac67dfc4d89b829f545f1407cc4285719eebbb5eb46f834debe034bf07a"
Sep 14 07:57:35.999551 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:35.999503    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feabcac67dfc4d89b829f545f1407cc4285719eebbb5eb46f834debe034bf07a"} err="failed to get container status \"feabcac67dfc4d89b829f545f1407cc4285719eebbb5eb46f834debe034bf07a\": rpc error: code = NotFound desc = could not find container \"feabcac67dfc4d89b829f545f1407cc4285719eebbb5eb46f834debe034bf07a\": container with ID starting with feabcac67dfc4d89b829f545f1407cc4285719eebbb5eb46f834debe034bf07a not found: ID does not exist"
Sep 14 07:57:38.857302 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:38.857254    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:38.960794 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:38.960766    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:41.259514 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:41.259481    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:41.360177 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:41.360147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:42.232762 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:42.232713    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"]
Sep 14 07:57:42.232964 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:42.232917    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8" podUID="f6ad5b8b-fe35-46bd-8f67-64a946091b41" containerName="manager" containerID="cri-o://05fdbb82d5f9b5a333b087f5f0686e83df2e2a7f6954425ff22c3a16584c720e" gracePeriod=10
Sep 14 07:57:42.235218 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:42.235186    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"
Sep 14 07:57:42.477298 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:42.477275    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"
Sep 14 07:57:42.595379 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:42.595319    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6ad5b8b-fe35-46bd-8f67-64a946091b41-extensions-socket-volume\") pod \"f6ad5b8b-fe35-46bd-8f67-64a946091b41\" (UID: \"f6ad5b8b-fe35-46bd-8f67-64a946091b41\") "
Sep 14 07:57:42.595379 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:42.595351    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zq6p6\" (UniqueName: \"kubernetes.io/projected/f6ad5b8b-fe35-46bd-8f67-64a946091b41-kube-api-access-zq6p6\") pod \"f6ad5b8b-fe35-46bd-8f67-64a946091b41\" (UID: \"f6ad5b8b-fe35-46bd-8f67-64a946091b41\") "
Sep 14 07:57:42.595766 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:42.595744    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6ad5b8b-fe35-46bd-8f67-64a946091b41-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f6ad5b8b-fe35-46bd-8f67-64a946091b41" (UID: "f6ad5b8b-fe35-46bd-8f67-64a946091b41"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:57:42.597454 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:42.597428    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6ad5b8b-fe35-46bd-8f67-64a946091b41-kube-api-access-zq6p6" (OuterVolumeSpecName: "kube-api-access-zq6p6") pod "f6ad5b8b-fe35-46bd-8f67-64a946091b41" (UID: "f6ad5b8b-fe35-46bd-8f67-64a946091b41"). InnerVolumeSpecName "kube-api-access-zq6p6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:57:42.696868 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:42.696841    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6ad5b8b-fe35-46bd-8f67-64a946091b41-extensions-socket-volume\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 07:57:42.696868 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:42.696864    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq6p6\" (UniqueName: \"kubernetes.io/projected/f6ad5b8b-fe35-46bd-8f67-64a946091b41-kube-api-access-zq6p6\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 07:57:43.022971 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:43.022939    2576 generic.go:356] "Generic (PLEG): container finished" podID="f6ad5b8b-fe35-46bd-8f67-64a946091b41" containerID="05fdbb82d5f9b5a333b087f5f0686e83df2e2a7f6954425ff22c3a16584c720e" exitCode=0
Sep 14 07:57:43.023117 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:43.022989    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"
Sep 14 07:57:43.023117 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:43.023027    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8" event={"ID":"f6ad5b8b-fe35-46bd-8f67-64a946091b41","Type":"ContainerDied","Data":"05fdbb82d5f9b5a333b087f5f0686e83df2e2a7f6954425ff22c3a16584c720e"}
Sep 14 07:57:43.023117 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:43.023098    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8" event={"ID":"f6ad5b8b-fe35-46bd-8f67-64a946091b41","Type":"ContainerDied","Data":"517750ebc07899cf69c3fdc706270e7e7b089691aa9d09cc5051a4f9234906dc"}
Sep 14 07:57:43.023310 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:43.023129    2576 scope.go:117] "RemoveContainer" containerID="05fdbb82d5f9b5a333b087f5f0686e83df2e2a7f6954425ff22c3a16584c720e"
Sep 14 07:57:43.031700 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:43.031683    2576 scope.go:117] "RemoveContainer" containerID="05fdbb82d5f9b5a333b087f5f0686e83df2e2a7f6954425ff22c3a16584c720e"
Sep 14 07:57:43.031975 ip-10-0-133-123 kubenswrapper[2576]: E0914 07:57:43.031957    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05fdbb82d5f9b5a333b087f5f0686e83df2e2a7f6954425ff22c3a16584c720e\": container with ID starting with 05fdbb82d5f9b5a333b087f5f0686e83df2e2a7f6954425ff22c3a16584c720e not found: ID does not exist" containerID="05fdbb82d5f9b5a333b087f5f0686e83df2e2a7f6954425ff22c3a16584c720e"
Sep 14 07:57:43.032013 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:43.031985    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05fdbb82d5f9b5a333b087f5f0686e83df2e2a7f6954425ff22c3a16584c720e"} err="failed to get container status \"05fdbb82d5f9b5a333b087f5f0686e83df2e2a7f6954425ff22c3a16584c720e\": rpc error: code = NotFound desc = could not find container \"05fdbb82d5f9b5a333b087f5f0686e83df2e2a7f6954425ff22c3a16584c720e\": container with ID starting with 05fdbb82d5f9b5a333b087f5f0686e83df2e2a7f6954425ff22c3a16584c720e not found: ID does not exist"
Sep 14 07:57:43.045734 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:43.045710    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"]
Sep 14 07:57:43.054944 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:43.054923    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2td8"]
Sep 14 07:57:43.838784 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:43.838751    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6ad5b8b-fe35-46bd-8f67-64a946091b41" path="/var/lib/kubelet/pods/f6ad5b8b-fe35-46bd-8f67-64a946091b41/volumes"
Sep 14 07:57:47.066005 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:47.065951    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:47.165855 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:47.165819    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:47.266526 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:47.266490    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:47.369476 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:47.369447    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:52.233305 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:52.233270    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:52.341627 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:52.341588    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:52.443168 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:52.443141    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:58.758223 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.758177    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"]
Sep 14 07:57:58.758606 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.758587    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6ad5b8b-fe35-46bd-8f67-64a946091b41" containerName="manager"
Sep 14 07:57:58.758664 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.758609    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6ad5b8b-fe35-46bd-8f67-64a946091b41" containerName="manager"
Sep 14 07:57:58.758664 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.758626    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="031fbc82-0bc0-47ed-9ed1-984dcefe3d64" containerName="manager"
Sep 14 07:57:58.758664 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.758636    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="031fbc82-0bc0-47ed-9ed1-984dcefe3d64" containerName="manager"
Sep 14 07:57:58.758794 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.758698    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="031fbc82-0bc0-47ed-9ed1-984dcefe3d64" containerName="manager"
Sep 14 07:57:58.758794 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.758707    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6ad5b8b-fe35-46bd-8f67-64a946091b41" containerName="manager"
Sep 14 07:57:58.762928 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.762908    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:58.765670 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.765648    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-zvctp\""
Sep 14 07:57:58.792065 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.792041    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"]
Sep 14 07:57:58.913842 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.913802    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:58.914011 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.913846    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znw96\" (UniqueName: \"kubernetes.io/projected/11002022-86a1-4120-812b-41b43fd80b36-kube-api-access-znw96\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:58.914011 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.913877    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:58.914011 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.913895    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:58.914011 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.913942    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:58.914011 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.913976    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11002022-86a1-4120-812b-41b43fd80b36-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:58.914340 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.914016    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11002022-86a1-4120-812b-41b43fd80b36-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:58.914340 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.914146    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:58.914340 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.914176    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11002022-86a1-4120-812b-41b43fd80b36-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:58.972156 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:58.972132    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:59.015177 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.015103    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znw96\" (UniqueName: \"kubernetes.io/projected/11002022-86a1-4120-812b-41b43fd80b36-kube-api-access-znw96\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.015177 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.015142    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.015177 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.015166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.015436 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.015193    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.015436 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.015240    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11002022-86a1-4120-812b-41b43fd80b36-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.015436 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.015268    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11002022-86a1-4120-812b-41b43fd80b36-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.015436 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.015339    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.015436 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.015370    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11002022-86a1-4120-812b-41b43fd80b36-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.015685 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.015550    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.015685 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.015601    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.015685 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.015632    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.015825 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.015813    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.015967 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.015944    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.016017 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.016001    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11002022-86a1-4120-812b-41b43fd80b36-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.017541 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.017510    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11002022-86a1-4120-812b-41b43fd80b36-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.017737 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.017721    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11002022-86a1-4120-812b-41b43fd80b36-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.026766 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.026743    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11002022-86a1-4120-812b-41b43fd80b36-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.026982 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.026965    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znw96\" (UniqueName: \"kubernetes.io/projected/11002022-86a1-4120-812b-41b43fd80b36-kube-api-access-znw96\") pod \"maas-default-gateway-openshift-default-5ccb968c66-fct8p\" (UID: \"11002022-86a1-4120-812b-41b43fd80b36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.072972 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.072950    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:57:59.073873 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.073853    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:59.347338 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.347314    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"]
Sep 14 07:57:59.348973 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:57:59.348948    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11002022_86a1_4120_812b_41b43fd80b36.slice/crio-928f1e2210bd651a72069fb652e4c968ce9460e23a1d5851f6d2e8bb87cc633b WatchSource:0}: Error finding container 928f1e2210bd651a72069fb652e4c968ce9460e23a1d5851f6d2e8bb87cc633b: Status 404 returned error can't find the container with id 928f1e2210bd651a72069fb652e4c968ce9460e23a1d5851f6d2e8bb87cc633b
Sep 14 07:57:59.351019 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.350986    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 07:57:59.351086 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.351053    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 07:57:59.351086 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:57:59.351082    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 07:58:00.108596 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:00.108554    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p" event={"ID":"11002022-86a1-4120-812b-41b43fd80b36","Type":"ContainerStarted","Data":"fc9583dfe59b5bd8301312537c930c9e2f4f716bac04346b738cf974093faa30"}
Sep 14 07:58:00.109007 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:00.108621    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p" event={"ID":"11002022-86a1-4120-812b-41b43fd80b36","Type":"ContainerStarted","Data":"928f1e2210bd651a72069fb652e4c968ce9460e23a1d5851f6d2e8bb87cc633b"}
Sep 14 07:58:00.129101 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:00.129057    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p" podStartSLOduration=2.1290414699999998 podStartE2EDuration="2.12904147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:58:00.127766229 +0000 UTC m=+534.832304733" watchObservedRunningTime="2026-09-14 07:58:00.12904147 +0000 UTC m=+534.833579972"
Sep 14 07:58:01.073427 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:01.073393    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:58:01.078188 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:01.078159    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:58:01.113676 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:01.113652    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:58:01.114481 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:01.114466    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:58:05.892825 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:05.892793    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:05.947090 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:05.947060    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:14.379857 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.379820    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:14.504143 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.504111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:14.559517 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.559486    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:14.696110 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.696039    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 07:58:14.699629 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.699613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"
Sep 14 07:58:14.701678 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.701653    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 14 07:58:14.701678 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.701673    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rfj9r\""
Sep 14 07:58:14.708745 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.708723    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 07:58:14.719511 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.719481    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 07:58:14.845601 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.845578    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf5lh\" (UniqueName: \"kubernetes.io/projected/f0f6364d-e5b4-4899-8ebd-2a2824c84534-kube-api-access-sf5lh\") pod \"limitador-limitador-84994f7ffc-gdlnh\" (UID: \"f0f6364d-e5b4-4899-8ebd-2a2824c84534\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"
Sep 14 07:58:14.845712 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.845620    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f0f6364d-e5b4-4899-8ebd-2a2824c84534-config-file\") pod \"limitador-limitador-84994f7ffc-gdlnh\" (UID: \"f0f6364d-e5b4-4899-8ebd-2a2824c84534\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"
Sep 14 07:58:14.946464 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.946409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf5lh\" (UniqueName: \"kubernetes.io/projected/f0f6364d-e5b4-4899-8ebd-2a2824c84534-kube-api-access-sf5lh\") pod \"limitador-limitador-84994f7ffc-gdlnh\" (UID: \"f0f6364d-e5b4-4899-8ebd-2a2824c84534\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"
Sep 14 07:58:14.946464 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.946460    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f0f6364d-e5b4-4899-8ebd-2a2824c84534-config-file\") pod \"limitador-limitador-84994f7ffc-gdlnh\" (UID: \"f0f6364d-e5b4-4899-8ebd-2a2824c84534\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"
Sep 14 07:58:14.947085 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.947066    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f0f6364d-e5b4-4899-8ebd-2a2824c84534-config-file\") pod \"limitador-limitador-84994f7ffc-gdlnh\" (UID: \"f0f6364d-e5b4-4899-8ebd-2a2824c84534\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"
Sep 14 07:58:14.954409 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:14.954385    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf5lh\" (UniqueName: \"kubernetes.io/projected/f0f6364d-e5b4-4899-8ebd-2a2824c84534-kube-api-access-sf5lh\") pod \"limitador-limitador-84994f7ffc-gdlnh\" (UID: \"f0f6364d-e5b4-4899-8ebd-2a2824c84534\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"
Sep 14 07:58:15.010885 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:15.010860    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"
Sep 14 07:58:15.209174 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:15.209105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:15.280143 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:15.280061    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 07:58:15.282376 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:15.282312    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:15.283274 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:58:15.283242    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f6364d_e5b4_4899_8ebd_2a2824c84534.slice/crio-743010a90fd9a7d6ae82523670ee63387c6bec073642e5088a4f4e01a4148545 WatchSource:0}: Error finding container 743010a90fd9a7d6ae82523670ee63387c6bec073642e5088a4f4e01a4148545: Status 404 returned error can't find the container with id 743010a90fd9a7d6ae82523670ee63387c6bec073642e5088a4f4e01a4148545
Sep 14 07:58:16.189052 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:16.189015    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-gdlnh" event={"ID":"f0f6364d-e5b4-4899-8ebd-2a2824c84534","Type":"ContainerStarted","Data":"743010a90fd9a7d6ae82523670ee63387c6bec073642e5088a4f4e01a4148545"}
Sep 14 07:58:17.195858 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:17.195815    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-gdlnh" event={"ID":"f0f6364d-e5b4-4899-8ebd-2a2824c84534","Type":"ContainerStarted","Data":"b47ef56bdc14429f8ae77d2280e8b8471d522db6759eeab392742254ab78cd6f"}
Sep 14 07:58:17.196300 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:17.195885    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"
Sep 14 07:58:17.211516 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:17.211467    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-gdlnh" podStartSLOduration=2.08595418 podStartE2EDuration="3.211451058s" totalImagesPullingTime="1.125496878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:58:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:58:17.209049336 +0000 UTC m=+551.913587855" watchObservedRunningTime="2026-09-14 07:58:17.211451058 +0000 UTC m=+551.915989559"
Sep 14 07:58:28.202579 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:28.202540    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"
Sep 14 07:58:33.059975 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:33.059931    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:33.159135 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:33.159099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:33.650039 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:33.650004    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:42.969187 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:42.969145    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:43.077417 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:43.077340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:52.092091 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:52.092057    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:52.185933 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:52.185903    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:52.284187 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:52.284159    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:52.338327 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:52.338296    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:52.587475 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:52.587443    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:56.470333 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:56.470302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:56.569562 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:58:56.569530    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:59:26.641413 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:26.641377    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:59:28.713087 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:28.713010    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 07:59:28.902491 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:28.902453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:59:29.003305 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:29.003225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:59:30.255239 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:30.255190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:59:38.004640 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:38.004611    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:59:38.139115 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:38.139056    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:59:38.598827 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:38.598753    2576 generic.go:356] "Generic (PLEG): container finished" podID="11002022-86a1-4120-812b-41b43fd80b36" containerID="fc9583dfe59b5bd8301312537c930c9e2f4f716bac04346b738cf974093faa30" exitCode=0
Sep 14 07:59:38.598966 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:38.598828    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p" event={"ID":"11002022-86a1-4120-812b-41b43fd80b36","Type":"ContainerDied","Data":"fc9583dfe59b5bd8301312537c930c9e2f4f716bac04346b738cf974093faa30"}
Sep 14 07:59:38.599276 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:38.599257    2576 scope.go:117] "RemoveContainer" containerID="fc9583dfe59b5bd8301312537c930c9e2f4f716bac04346b738cf974093faa30"
Sep 14 07:59:38.600418 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:38.600382    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 07:59:38.600486 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:38.600467    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 07:59:38.600522 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:38.600504    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 07:59:39.606552 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:39.606519    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p" event={"ID":"11002022-86a1-4120-812b-41b43fd80b36","Type":"ContainerStarted","Data":"287cfd2d3215a85a6ea3616b3d301c138b36207b38aaca24ef7437e1b011f72a"}
Sep 14 07:59:40.074295 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:40.074187    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:59:40.074443 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:40.074368    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-fct8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 07:59:40.074443 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:40.074411    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p" podUID="11002022-86a1-4120-812b-41b43fd80b36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 07:59:41.074356 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:41.074320    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-fct8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 07:59:41.074728 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:41.074405    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p" podUID="11002022-86a1-4120-812b-41b43fd80b36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 07:59:42.073533 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:42.073495    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-fct8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 07:59:42.073693 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:42.073561    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p" podUID="11002022-86a1-4120-812b-41b43fd80b36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 07:59:43.073809 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:43.073726    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-fct8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 07:59:43.073809 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:43.073795    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p" podUID="11002022-86a1-4120-812b-41b43fd80b36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 07:59:44.073375 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:44.073342    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-fct8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 07:59:44.073541 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:44.073396    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p" podUID="11002022-86a1-4120-812b-41b43fd80b36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 07:59:45.010374 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:45.010343    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:59:45.076821 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:45.076793    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:59:45.077085 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:45.077061    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:59:45.077737 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:45.077716    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-fct8p"
Sep 14 07:59:45.387289 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:45.387252    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 07:59:51.378303 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.378267    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"]
Sep 14 07:59:51.381484 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.381464    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.383970 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.383949    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 14 07:59:51.384790 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.384763    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 14 07:59:51.384790 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.384777    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dsj5g\""
Sep 14 07:59:51.384947 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.384805    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 14 07:59:51.388446 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.388425    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"]
Sep 14 07:59:51.412049 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.412021    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.412176 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.412052    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.412176 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.412070    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.412176 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.412161    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.412348 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.412270    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6j8w\" (UniqueName: \"kubernetes.io/projected/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-kube-api-access-d6j8w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.412348 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.412299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.417697 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.417676    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:59:51.513786 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.513748    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6j8w\" (UniqueName: \"kubernetes.io/projected/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-kube-api-access-d6j8w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.513936 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.513789    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.513936 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.513860    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.513936 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.513889    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.513936 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.513912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.514132 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.513973    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.514317 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.514291    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.514401 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.514300    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.514401 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.514362    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.516179 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.516152    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.516468 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.516451    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.521774 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.521750    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6j8w\" (UniqueName: \"kubernetes.io/projected/624c18ed-3aef-4bb2-ad28-2c4ec5e3f308-kube-api-access-d6j8w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9498\" (UID: \"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.692964 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.692879    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 07:59:51.965540 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.965511    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"]
Sep 14 07:59:51.967032 ip-10-0-133-123 kubenswrapper[2576]: W0914 07:59:51.967005    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod624c18ed_3aef_4bb2_ad28_2c4ec5e3f308.slice/crio-9ab78d64248caddd654e6420124ab1857c891d4d56eaae3ff20c01eb5d06ed42 WatchSource:0}: Error finding container 9ab78d64248caddd654e6420124ab1857c891d4d56eaae3ff20c01eb5d06ed42: Status 404 returned error can't find the container with id 9ab78d64248caddd654e6420124ab1857c891d4d56eaae3ff20c01eb5d06ed42
Sep 14 07:59:51.968292 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:51.968271    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:59:52.184688 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:52.184654    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 07:59:52.674165 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:52.674086    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498" event={"ID":"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308","Type":"ContainerStarted","Data":"9ab78d64248caddd654e6420124ab1857c891d4d56eaae3ff20c01eb5d06ed42"}
Sep 14 07:59:53.680562 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:53.680526    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498" event={"ID":"624c18ed-3aef-4bb2-ad28-2c4ec5e3f308","Type":"ContainerStarted","Data":"a084bb4beb7b8fdeb7e8c1ea788266576f2808e0d0ea33f00e7a448936d727ba"}
Sep 14 07:59:53.698513 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:53.698466    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498" podStartSLOduration=1.709945509 podStartE2EDuration="2.698449361s" totalImagesPullingTime="988.503852ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:59:53.696366164 +0000 UTC m=+648.400904684" watchObservedRunningTime="2026-09-14 07:59:53.698449361 +0000 UTC m=+648.402987857"
Sep 14 07:59:54.107124 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:54.107098    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 07:59:54.987802 ip-10-0-133-123 kubenswrapper[2576]: I0914 07:59:54.987772    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:00:01.693247 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:00:01.693204    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 08:00:01.693771 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:00:01.693284    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 08:00:01.705508 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:00:01.705480    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 08:00:01.726994 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:00:01.726969    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9498"
Sep 14 08:00:05.810508 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:00:05.810469    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:07.000181 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:00:07.000141    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:00:09.668961 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:00:09.668932    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:10.898477 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:00:10.898445    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:00:15.978929 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:00:15.978902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:19.404549 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:00:19.404519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:21.091223 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:00:21.091186    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:00:30.911726 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:00:30.911689    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:32.529729 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:00:32.529689    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:32.995035 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:00:32.995002    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:01:24.941517 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:01:24.941482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:31.305157 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:31.305081    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:31.409452 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:31.409423    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:32.388371 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:32.388330    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:32.487058 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:32.487030    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:37.924976 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:37.924938    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:38.025728 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:38.025696    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:48.354709 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.354667    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"]
Sep 14 08:02:48.358544 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.358522    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.360708 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.360685    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-cd7582-openshift-default-dockercfg-ttnps\""
Sep 14 08:02:48.369497 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.369472    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"]
Sep 14 08:02:48.418597 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.418569    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"]
Sep 14 08:02:48.422224 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.422186    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.425282 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.425252    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e7b0a8-openshift-default-dockercfg-jzv5v\""
Sep 14 08:02:48.439135 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.439113    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"]
Sep 14 08:02:48.446830 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.446800    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-workload-certs\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.446929 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.446863    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-data\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.446929 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.446890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m86p8\" (UniqueName: \"kubernetes.io/projected/db74fb53-47cb-4e85-9aab-c7feadb5f64f-kube-api-access-m86p8\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.447004 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.446978    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-workload-socket\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.447038 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.447009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-token\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.447071 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.447042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-credential-socket\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.447071 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.447057    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-envoy\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.447132 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.447084    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-podinfo\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.447204 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.447188    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istiod-ca-cert\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.485727 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.485700    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"]
Sep 14 08:02:48.489085 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.489064    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.491312 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.491285    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-831f8e-openshift-default-dockercfg-24zxg\""
Sep 14 08:02:48.500415 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.500390    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"]
Sep 14 08:02:48.548021 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.547993    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-workload-certs\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.548139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548028    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-data\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.548139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548046    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m86p8\" (UniqueName: \"kubernetes.io/projected/db74fb53-47cb-4e85-9aab-c7feadb5f64f-kube-api-access-m86p8\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.548139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-workload-certs\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.548139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548112    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-credential-socket\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.548356 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548137    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-envoy\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.548356 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548162    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-podinfo\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.548356 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548309    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-workload-socket\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.548479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-workload-socket\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.548479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548383    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-data\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.548479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548398    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-token\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.548479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548410    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-workload-certs\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.548479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548418    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-data\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.548479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548435    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-token\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.548479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548458    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-data\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.548479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548472    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-token\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.548758 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d86n7\" (UniqueName: \"kubernetes.io/projected/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-kube-api-access-d86n7\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.548758 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548558    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-workload-socket\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.548758 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548587    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-credential-socket\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.548758 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-envoy\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.548758 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548618    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-podinfo\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.548758 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548629    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-workload-socket\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.548758 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548640    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgxmv\" (UniqueName: \"kubernetes.io/projected/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-kube-api-access-pgxmv\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.548758 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548694    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-envoy\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.548758 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548731    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-credential-socket\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.549047 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548770    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-podinfo\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.549047 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548806    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-credential-socket\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.549047 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548813    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istiod-ca-cert\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.549047 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548841    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-workload-certs\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.549047 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548870    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istiod-ca-cert\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.549047 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.548908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istiod-ca-cert\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.549512 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.549495    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istiod-ca-cert\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.550942 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.550917    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-envoy\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.551259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.551240    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-podinfo\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.556189 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.556166    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-token\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.556334 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.556319    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m86p8\" (UniqueName: \"kubernetes.io/projected/db74fb53-47cb-4e85-9aab-c7feadb5f64f-kube-api-access-m86p8\") pod \"e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") " pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.649888 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.649799    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istiod-ca-cert\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.649888 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.649833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-workload-certs\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.649888 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.649852    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istiod-ca-cert\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.650154 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.649912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-workload-certs\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.650154 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.649936    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-credential-socket\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.650154 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.649956    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-envoy\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.650154 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.649980    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-podinfo\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.650154 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650022    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-workload-socket\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.650154 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-data\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.650154 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-token\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.650154 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650098    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-data\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.650154 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650121    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-token\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.650710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650178    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d86n7\" (UniqueName: \"kubernetes.io/projected/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-kube-api-access-d86n7\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.650710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650236    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-workload-socket\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.650710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650282    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgxmv\" (UniqueName: \"kubernetes.io/projected/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-kube-api-access-pgxmv\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.650710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650314    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-envoy\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.650710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-credential-socket\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.650710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650368    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-podinfo\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.650710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650472    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-credential-socket\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.650710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650621    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-workload-certs\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.650710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650699    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istiod-ca-cert\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.651139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650719    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-workload-certs\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.651139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.650869    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istiod-ca-cert\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.651139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.651005    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-data\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.651139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.651029    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-data\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.651374 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.651330    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-workload-socket\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.651474 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.651452    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-credential-socket\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.651578 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.651556    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-workload-socket\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.652592 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.652570    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-envoy\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.653386 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.653357    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-podinfo\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.653488 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.653399    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-envoy\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.653566 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.653544    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-podinfo\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.658004 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.657976    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-token\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.658097 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.658028    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgxmv\" (UniqueName: \"kubernetes.io/projected/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-kube-api-access-pgxmv\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.658270 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.658251    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d86n7\" (UniqueName: \"kubernetes.io/projected/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-kube-api-access-d86n7\") pod \"e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") " pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.658748 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.658732    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-token\") pod \"e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") " pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.670770 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.670745    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:48.734638 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.734613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:48.802820 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.802778    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:48.948587 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.948558    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"]
Sep 14 08:02:48.951270 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:02:48.951231    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb74fb53_47cb_4e85_9aab_c7feadb5f64f.slice/crio-c345281a4b406a647d07a019e66e58367c721db9665603b02d2e1f533cef7b5b WatchSource:0}: Error finding container c345281a4b406a647d07a019e66e58367c721db9665603b02d2e1f533cef7b5b: Status 404 returned error can't find the container with id c345281a4b406a647d07a019e66e58367c721db9665603b02d2e1f533cef7b5b
Sep 14 08:02:48.953509 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.953478    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:02:48.953583 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.953545    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:02:48.953583 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:48.953573    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:02:49.016362 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.016339    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"]
Sep 14 08:02:49.017389 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.017366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:49.017823 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:02:49.017799    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bc1f23c_a74a_49e9_8a52_5fbd6e9fdf08.slice/crio-ea65ef8113977ec4b75231e29d2cf4c644bf43b9da58ecdb288484d14e7a3067 WatchSource:0}: Error finding container ea65ef8113977ec4b75231e29d2cf4c644bf43b9da58ecdb288484d14e7a3067: Status 404 returned error can't find the container with id ea65ef8113977ec4b75231e29d2cf4c644bf43b9da58ecdb288484d14e7a3067
Sep 14 08:02:49.020116 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.020086    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:02:49.020332 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.020148    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:02:49.020332 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.020178    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:02:49.086775 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.086745    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"]
Sep 14 08:02:49.088221 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:02:49.088182    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod553aeccf_0f9f_4cde_9b5c_3b6bd897da0d.slice/crio-c81124c6de28febbe5350c94736931d017c4ea3b1f78b48bf16dc13efbccade9 WatchSource:0}: Error finding container c81124c6de28febbe5350c94736931d017c4ea3b1f78b48bf16dc13efbccade9: Status 404 returned error can't find the container with id c81124c6de28febbe5350c94736931d017c4ea3b1f78b48bf16dc13efbccade9
Sep 14 08:02:49.090112 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.090090    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:49.090705 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.090668    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:02:49.090770 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.090754    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:02:49.090817 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.090795    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:02:49.490665 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.490631    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" event={"ID":"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08","Type":"ContainerStarted","Data":"92864bf89aeaf9ebf2fbed8f3e7f51e673f50d3d94ab10298e49a6e2273b8fcd"}
Sep 14 08:02:49.491121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.490691    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" event={"ID":"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08","Type":"ContainerStarted","Data":"ea65ef8113977ec4b75231e29d2cf4c644bf43b9da58ecdb288484d14e7a3067"}
Sep 14 08:02:49.492276 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.492200    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" event={"ID":"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d","Type":"ContainerStarted","Data":"f752457f1bb0ffcb31a9f5967320305c45df82d61503edcc503faa48af675dac"}
Sep 14 08:02:49.492276 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.492259    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" event={"ID":"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d","Type":"ContainerStarted","Data":"c81124c6de28febbe5350c94736931d017c4ea3b1f78b48bf16dc13efbccade9"}
Sep 14 08:02:49.494108 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.494083    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" event={"ID":"db74fb53-47cb-4e85-9aab-c7feadb5f64f","Type":"ContainerStarted","Data":"3d4693a3c0931d55e70ea78e00e155c4e4e605566a6775bb4284320ee2a631ea"}
Sep 14 08:02:49.494231 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.494129    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" event={"ID":"db74fb53-47cb-4e85-9aab-c7feadb5f64f","Type":"ContainerStarted","Data":"c345281a4b406a647d07a019e66e58367c721db9665603b02d2e1f533cef7b5b"}
Sep 14 08:02:49.515963 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.515905    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" podStartSLOduration=1.515890731 podStartE2EDuration="1.515890731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:49.511617581 +0000 UTC m=+824.216156083" watchObservedRunningTime="2026-09-14 08:02:49.515890731 +0000 UTC m=+824.220429236"
Sep 14 08:02:49.524289 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.524264    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:49.536704 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.536659    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" podStartSLOduration=1.536646877 podStartE2EDuration="1.536646877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:49.535119213 +0000 UTC m=+824.239657717" watchObservedRunningTime="2026-09-14 08:02:49.536646877 +0000 UTC m=+824.241185380"
Sep 14 08:02:49.562226 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.558620    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" podStartSLOduration=1.5585943370000002 podStartE2EDuration="1.558594337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:49.556487043 +0000 UTC m=+824.261025568" watchObservedRunningTime="2026-09-14 08:02:49.558594337 +0000 UTC m=+824.263132840"
Sep 14 08:02:49.591449 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.591418    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:49.671659 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.671622    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:49.735130 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.735095    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:49.802914 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:49.802849    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:50.040076 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:50.040045    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:50.676322 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:50.676291    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:50.739939 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:50.739907    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:50.808329 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:50.808306    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:51.505286 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:51.505246    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:51.505647 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:51.505625    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:51.505823 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:51.505800    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:51.506343 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:51.506321    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:02:51.506528 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:51.506509    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:02:51.506616 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:51.506561    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:02:51.552438 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:51.551686    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:51.638125 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:51.638090    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:51.710989 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:51.710955    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:02:51.736753 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:51.736727    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:52.784692 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:52.784653    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:52.876458 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:52.876428    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:53.034009 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:53.033973    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:53.387784 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:53.387750    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:53.494689 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:53.494655    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:53.786926 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:53.786850    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:53.887024 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:53.886985    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:53.986974 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:53.986942    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:54.240617 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:54.240588    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:54.332546 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:54.332510    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:54.900274 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:54.900197    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"]
Sep 14 08:02:54.905441 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:54.905418    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:54.908089 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:54.908047    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-831f8e-serving-cert\""
Sep 14 08:02:54.908089 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:54.908066    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zwz44\""
Sep 14 08:02:54.908383 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:54.908099    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:02:54.909012 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:54.908988    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 08:02:54.909198 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:54.909184    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 08:02:54.918252 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:54.918225    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"]
Sep 14 08:02:54.941119 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:54.941091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:55.008468 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.008432    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-maas-api-tls\") pod \"maas-api-e2e-worker-w0-831f8e-cc966c997-szks6\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:55.008468 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.008471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-metrics-certs\") pod \"maas-api-e2e-worker-w0-831f8e-cc966c997-szks6\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:55.008674 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.008498    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50945a1e-fa44-49c7-b141-7e29449f1962-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-831f8e-cc966c997-szks6\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:55.008674 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.008564    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rklqd\" (UniqueName: \"kubernetes.io/projected/50945a1e-fa44-49c7-b141-7e29449f1962-kube-api-access-rklqd\") pod \"maas-api-e2e-worker-w0-831f8e-cc966c997-szks6\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:55.056634 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.056603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:55.109623 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.109599    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50945a1e-fa44-49c7-b141-7e29449f1962-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-831f8e-cc966c997-szks6\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:55.109741 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.109637    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rklqd\" (UniqueName: \"kubernetes.io/projected/50945a1e-fa44-49c7-b141-7e29449f1962-kube-api-access-rklqd\") pod \"maas-api-e2e-worker-w0-831f8e-cc966c997-szks6\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:55.109741 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.109707    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-maas-api-tls\") pod \"maas-api-e2e-worker-w0-831f8e-cc966c997-szks6\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:55.109741 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.109727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-metrics-certs\") pod \"maas-api-e2e-worker-w0-831f8e-cc966c997-szks6\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:55.109897 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:02:55.109826    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-serving-cert: secret "maas-api-e2e-worker-w0-831f8e-serving-cert" not found
Sep 14 08:02:55.109952 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:02:55.109907    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-maas-api-tls podName:50945a1e-fa44-49c7-b141-7e29449f1962 nodeName:}" failed. No retries permitted until 2026-09-14 08:02:55.609887513 +0000 UTC m=+830.314426001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-maas-api-tls") pod "maas-api-e2e-worker-w0-831f8e-cc966c997-szks6" (UID: "50945a1e-fa44-49c7-b141-7e29449f1962") : secret "maas-api-e2e-worker-w0-831f8e-serving-cert" not found
Sep 14 08:02:55.110373 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.110352    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50945a1e-fa44-49c7-b141-7e29449f1962-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-831f8e-cc966c997-szks6\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:55.112271 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.112252    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-metrics-certs\") pod \"maas-api-e2e-worker-w0-831f8e-cc966c997-szks6\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:55.127627 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.127603    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rklqd\" (UniqueName: \"kubernetes.io/projected/50945a1e-fa44-49c7-b141-7e29449f1962-kube-api-access-rklqd\") pod \"maas-api-e2e-worker-w0-831f8e-cc966c997-szks6\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:55.240200 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.240122    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:55.460121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.460093    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:55.557992 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.557921    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:55.614144 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.614106    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-maas-api-tls\") pod \"maas-api-e2e-worker-w0-831f8e-cc966c997-szks6\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:55.616620 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.616581    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-maas-api-tls\") pod \"maas-api-e2e-worker-w0-831f8e-cc966c997-szks6\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:55.819999 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.819901    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:55.945506 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:55.945467    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:56.095730 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:56.095705    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"]
Sep 14 08:02:56.097333 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:02:56.097303    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50945a1e_fa44_49c7_b141_7e29449f1962.slice/crio-75fe6136e75f89bb71e3a9bec3190710e24858a9543a9455cc382eb8285f905e WatchSource:0}: Error finding container 75fe6136e75f89bb71e3a9bec3190710e24858a9543a9455cc382eb8285f905e: Status 404 returned error can't find the container with id 75fe6136e75f89bb71e3a9bec3190710e24858a9543a9455cc382eb8285f905e
Sep 14 08:02:56.498107 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:56.498073    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:56.532388 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:56.532353    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6" event={"ID":"50945a1e-fa44-49c7-b141-7e29449f1962","Type":"ContainerStarted","Data":"75fe6136e75f89bb71e3a9bec3190710e24858a9543a9455cc382eb8285f905e"}
Sep 14 08:02:56.558547 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:56.558488    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:56.648896 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:56.648861    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:56.749692 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:56.749605    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:56.981520 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:56.979947    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"]
Sep 14 08:02:56.984343 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:56.984318    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:56.987437 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:56.987337    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9ab35f8f-openshift-default-dockercfg-qp4bq\""
Sep 14 08:02:57.001541 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.001476    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"]
Sep 14 08:02:57.033699 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.033633    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:57.131368 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.131325    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-podinfo\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.131569 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.131395    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-workload-socket\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.131569 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.131507    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-data\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.131694 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.131568    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-envoy\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.131694 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.131623    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxlcs\" (UniqueName: \"kubernetes.io/projected/8f528660-338e-4365-a5f8-c8aafef9bf5d-kube-api-access-xxlcs\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.131694 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.131657    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-workload-certs\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.131694 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.131688    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-credential-socket\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.131894 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.131723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f528660-338e-4365-a5f8-c8aafef9bf5d-istiod-ca-cert\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.131894 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.131798    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-token\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.233310 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.233266    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-envoy\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.233495 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.233334    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxlcs\" (UniqueName: \"kubernetes.io/projected/8f528660-338e-4365-a5f8-c8aafef9bf5d-kube-api-access-xxlcs\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.233495 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.233371    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-workload-certs\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.233495 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.233405    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-credential-socket\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.233495 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.233442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f528660-338e-4365-a5f8-c8aafef9bf5d-istiod-ca-cert\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.233720 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.233496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-token\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.233720 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.233539    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-podinfo\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.233720 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.233585    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-workload-socket\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.233720 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.233638    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-data\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.234287 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.234259    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-data\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.234502 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.234480    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-workload-certs\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.234770 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.234748    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-credential-socket\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.234835 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.234774    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-workload-socket\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.235287 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.235266    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f528660-338e-4365-a5f8-c8aafef9bf5d-istiod-ca-cert\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.235603 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.235581    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:57.237571 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.237528    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-podinfo\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.237700 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.237678    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-envoy\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.242034 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.241991    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-token\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.242590 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.242560    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxlcs\" (UniqueName: \"kubernetes.io/projected/8f528660-338e-4365-a5f8-c8aafef9bf5d-kube-api-access-xxlcs\") pod \"e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") " pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.303635 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.302776    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:02:57.343096 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:57.343068    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:58.546104 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:58.546075    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6" event={"ID":"50945a1e-fa44-49c7-b141-7e29449f1962","Type":"ContainerStarted","Data":"6ebc0578fe27ee3f37276cea126d35017ba2b15a994ac8b99e15fde7e7516f13"}
Sep 14 08:02:58.546457 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:58.546200    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:02:58.567084 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:58.567041    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6" podStartSLOduration=2.212294102 podStartE2EDuration="4.567027294s" totalImagesPullingTime="2.354733192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:58.564948905 +0000 UTC m=+833.269487400" watchObservedRunningTime="2026-09-14 08:02:58.567027294 +0000 UTC m=+833.271565797"
Sep 14 08:02:58.668981 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:58.668954    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"]
Sep 14 08:02:58.671020 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:02:58.670992    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f528660_338e_4365_a5f8_c8aafef9bf5d.slice/crio-087ffd75d70ff96272bd89ff5d10744eb8ae63e1f297c98721a9ba69b78ad4f3 WatchSource:0}: Error finding container 087ffd75d70ff96272bd89ff5d10744eb8ae63e1f297c98721a9ba69b78ad4f3: Status 404 returned error can't find the container with id 087ffd75d70ff96272bd89ff5d10744eb8ae63e1f297c98721a9ba69b78ad4f3
Sep 14 08:02:58.673800 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:58.673770    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:02:58.673865 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:58.673832    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:02:58.673908 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:58.673866    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:02:59.552513 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:59.552470    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh" event={"ID":"8f528660-338e-4365-a5f8-c8aafef9bf5d","Type":"ContainerStarted","Data":"d9b85bfabd2a6a8233d44fc1e8395738f58cbf667aa6434e464fa878f33d7ec8"}
Sep 14 08:02:59.552937 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:02:59.552538    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh" event={"ID":"8f528660-338e-4365-a5f8-c8aafef9bf5d","Type":"ContainerStarted","Data":"087ffd75d70ff96272bd89ff5d10744eb8ae63e1f297c98721a9ba69b78ad4f3"}
Sep 14 08:03:00.029808 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.029774    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:00.257791 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.257731    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:00.304365 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.304199    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:03:00.310031 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.310005    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:03:00.317731 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.317684    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh" podStartSLOduration=4.317669501 podStartE2EDuration="4.317669501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:59.577077164 +0000 UTC m=+834.281615667" watchObservedRunningTime="2026-09-14 08:03:00.317669501 +0000 UTC m=+835.022208003"
Sep 14 08:03:00.318079 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.318062    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"]
Sep 14 08:03:00.336073 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.336036    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.337355 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.337327    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"]
Sep 14 08:03:00.342422 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.342399    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-885430-openshift-default-dockercfg-gs7xg\""
Sep 14 08:03:00.370237 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.369888    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:00.467232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.464738    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-data\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.467232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.464830    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-token\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.467232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.464861    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-credential-socket\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.467232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.464913    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-workload-socket\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.467232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.464962    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-podinfo\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.467232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.464999    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/271d49fc-4f91-466f-a1a3-800284dfa5d7-istiod-ca-cert\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.467232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.465035    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54fvz\" (UniqueName: \"kubernetes.io/projected/271d49fc-4f91-466f-a1a3-800284dfa5d7-kube-api-access-54fvz\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.467232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.465065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-envoy\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.467232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.465107    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-workload-certs\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.467232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.466548    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:00.560847 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.560752    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:03:00.566594 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.566558    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-token\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.566759 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.566610    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-credential-socket\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.566759 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.566659    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-workload-socket\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.566759 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.566704    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-podinfo\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.566759 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.566740    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/271d49fc-4f91-466f-a1a3-800284dfa5d7-istiod-ca-cert\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.566961 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.566775    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54fvz\" (UniqueName: \"kubernetes.io/projected/271d49fc-4f91-466f-a1a3-800284dfa5d7-kube-api-access-54fvz\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.566961 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.566801    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-envoy\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.566961 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.566840    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-workload-certs\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.566961 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.566888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-data\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.567327 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.567304    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-data\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.567851 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.567819    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-credential-socket\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.568062 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.568040    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-workload-socket\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.569037 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.569015    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/271d49fc-4f91-466f-a1a3-800284dfa5d7-istiod-ca-cert\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.569296 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.569264    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-workload-certs\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.571182 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.571137    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-envoy\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.571292 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.571247    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-podinfo\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.574855 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.574827    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:03:00.576586 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.576564    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-token\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.578491 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.578445    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54fvz\" (UniqueName: \"kubernetes.io/projected/271d49fc-4f91-466f-a1a3-800284dfa5d7-kube-api-access-54fvz\") pod \"e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") " pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.669172 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.669143    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:00.911982 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:00.911948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:01.033839 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:01.033749    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:01.116978 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:01.116714    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:01.387471 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:01.385703    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"]
Sep 14 08:03:01.416530 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:01.416351    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:01.416660 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:01.416629    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:01.416725 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:01.416713    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:01.570306 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:01.570273    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" event={"ID":"271d49fc-4f91-466f-a1a3-800284dfa5d7","Type":"ContainerStarted","Data":"3b5aa9cc01e2c3734990eb931ca0f5704ea1050a1aed17a83696c28d2db2ee51"}
Sep 14 08:03:02.037071 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:02.036921    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:02.580203 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:02.580165    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" event={"ID":"271d49fc-4f91-466f-a1a3-800284dfa5d7","Type":"ContainerStarted","Data":"0e4f52e31112dc04b0253886a81fe9f99668226aa49724718574635562ec666c"}
Sep 14 08:03:02.672522 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:02.672473    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:02.672721 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:02.672614    2576 patch_prober.go:28] interesting pod/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 14 08:03:02.672721 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:02.672662    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 14 08:03:02.734272 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:02.734199    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:03.537233 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:03.534613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:03.636670 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:03.636636    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:03.680238 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:03.679488    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:03.702470 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:03.702407    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" podStartSLOduration=3.70238638 podStartE2EDuration="3.70238638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:02.603566845 +0000 UTC m=+837.308105350" watchObservedRunningTime="2026-09-14 08:03:03.70238638 +0000 UTC m=+838.406924884"
Sep 14 08:03:03.942360 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:03.942324    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:04.151232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:04.145400    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:04.246446 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:04.246311    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:04.564793 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:04.564693    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:03:04.613159 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:04.613110    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:04.614712 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:04.614690    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:04.661234 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:04.657020    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:04.760235 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:04.756717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:04.849286 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:04.848672    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:05.490576 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:05.490534    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:06.253177 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:06.253142    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:06.355678 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:06.355650    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:06.812351 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:06.812320    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:06.935970 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:06.935886    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:07.325305 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:07.325142    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:07.429893 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:07.429810    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:07.525463 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:07.525332    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:07.547432 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:07.547386    2576 patch_prober.go:28] interesting pod/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 14 08:03:07.547628 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:07.547460    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 14 08:03:07.548355 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:07.548331    2576 patch_prober.go:28] interesting pod/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 14 08:03:07.548466 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:07.548374    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 14 08:03:07.630524 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:07.630389    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:07.737983 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:07.737870    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:07.848287 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:07.848243    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:07.926546 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:07.926454    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:08.062711 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:08.062677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:08.103794 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:08.103754    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:03:08.349317 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:08.349229    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:09.043740 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:09.043700    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:09.677623 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:09.677569    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:10.358772 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:10.358741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:10.495143 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:10.495037    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:11.212049 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.212020    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:11.411121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.411040    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:11.511660 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.511633    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:11.657591 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.657559    2576 generic.go:356] "Generic (PLEG): container finished" podID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerID="f752457f1bb0ffcb31a9f5967320305c45df82d61503edcc503faa48af675dac" exitCode=0
Sep 14 08:03:11.657757 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.657635    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" event={"ID":"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d","Type":"ContainerDied","Data":"f752457f1bb0ffcb31a9f5967320305c45df82d61503edcc503faa48af675dac"}
Sep 14 08:03:11.658155 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.658136    2576 scope.go:117] "RemoveContainer" containerID="f752457f1bb0ffcb31a9f5967320305c45df82d61503edcc503faa48af675dac"
Sep 14 08:03:11.659331 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.659309    2576 generic.go:356] "Generic (PLEG): container finished" podID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerID="3d4693a3c0931d55e70ea78e00e155c4e4e605566a6775bb4284320ee2a631ea" exitCode=0
Sep 14 08:03:11.659419 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.659398    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" event={"ID":"db74fb53-47cb-4e85-9aab-c7feadb5f64f","Type":"ContainerDied","Data":"3d4693a3c0931d55e70ea78e00e155c4e4e605566a6775bb4284320ee2a631ea"}
Sep 14 08:03:11.659505 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.659391    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:11.659596 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.659574    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:11.659662 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.659632    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:11.659769 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.659752    2576 scope.go:117] "RemoveContainer" containerID="3d4693a3c0931d55e70ea78e00e155c4e4e605566a6775bb4284320ee2a631ea"
Sep 14 08:03:11.663271 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.661758    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:11.663271 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.661840    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:11.663271 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.661885    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:11.663271 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.662863    2576 generic.go:356] "Generic (PLEG): container finished" podID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerID="92864bf89aeaf9ebf2fbed8f3e7f51e673f50d3d94ab10298e49a6e2273b8fcd" exitCode=0
Sep 14 08:03:11.663271 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.662896    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" event={"ID":"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08","Type":"ContainerDied","Data":"92864bf89aeaf9ebf2fbed8f3e7f51e673f50d3d94ab10298e49a6e2273b8fcd"}
Sep 14 08:03:11.663271 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.663268    2576 scope.go:117] "RemoveContainer" containerID="92864bf89aeaf9ebf2fbed8f3e7f51e673f50d3d94ab10298e49a6e2273b8fcd"
Sep 14 08:03:11.664402 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.664377    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:11.664487 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.664450    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:11.664675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.664502    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:11.767671 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.767641    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:11.968149 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:11.968061    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:12.108793 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.108759    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"]
Sep 14 08:03:12.126468 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.126438    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"]
Sep 14 08:03:12.126829 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.126761    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.132781 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.132758    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-f67e60-openshift-default-dockercfg-r4cg8\""
Sep 14 08:03:12.272495 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.272410    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-credential-socket\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.272495 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.272461    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-workload-socket\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.272495 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.272496    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-token\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.273055 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.272514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-data\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.273055 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.272536    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b3d784e-2474-4eee-8f41-55de57b117f2-istiod-ca-cert\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.273055 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.272609    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-workload-certs\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.273055 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.272663    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-podinfo\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.273055 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.272685    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqx5f\" (UniqueName: \"kubernetes.io/projected/1b3d784e-2474-4eee-8f41-55de57b117f2-kube-api-access-pqx5f\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.273055 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.272720    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-envoy\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.374049 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.374009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-podinfo\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.374817 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.374790    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqx5f\" (UniqueName: \"kubernetes.io/projected/1b3d784e-2474-4eee-8f41-55de57b117f2-kube-api-access-pqx5f\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.375417 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.375397    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-envoy\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.375947 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.375929    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-credential-socket\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.376595 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.376445    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-credential-socket\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.376749 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.376734    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-workload-socket\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.377202 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.377097    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-workload-socket\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.377386 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.377370    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-token\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.377748 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.377729    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-data\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.378231 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.378195    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b3d784e-2474-4eee-8f41-55de57b117f2-istiod-ca-cert\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.379067 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.379051    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-workload-certs\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.379626 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.378981    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b3d784e-2474-4eee-8f41-55de57b117f2-istiod-ca-cert\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.379751 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.378113    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-data\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.379845 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.379449    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-envoy\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.379930 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.379465    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-workload-certs\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.380849 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.379865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-podinfo\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.386920 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.386896    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqx5f\" (UniqueName: \"kubernetes.io/projected/1b3d784e-2474-4eee-8f41-55de57b117f2-kube-api-access-pqx5f\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.389730 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.389708    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-token\") pod \"e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") " pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.470702 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.470672    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:12.721389 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.721272    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" event={"ID":"db74fb53-47cb-4e85-9aab-c7feadb5f64f","Type":"ContainerStarted","Data":"ff8eba2cb98b92d9fdd6ce618f8959810fbd6110b4b4041a853ed0ba552911fe"}
Sep 14 08:03:12.730417 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.725592    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" event={"ID":"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08","Type":"ContainerStarted","Data":"15d8eac17ce3e4923cabb2e6bf75c93a127b899f25299c7f2a1e3565f54b9598"}
Sep 14 08:03:12.753255 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.750432    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:03:12.753255 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.750545    2576 patch_prober.go:28] interesting pod/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 08:03:12.753255 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.750585    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 08:03:12.795338 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.794734    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" event={"ID":"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d","Type":"ContainerStarted","Data":"68cc4f943aa36c5748eb50a291f7db53139b5772b7698396aca84fa07b825c2d"}
Sep 14 08:03:12.806269 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.804467    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:03:12.821239 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.817341    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 14 08:03:12.821239 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.817415    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 14 08:03:12.883462 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.883170    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"]
Sep 14 08:03:12.914455 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.909848    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:12.914455 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.909944    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:12.914455 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:12.909993    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:13.124032 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:13.123992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:13.226288 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:13.225413    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:13.671412 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:13.671372    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:03:13.671923 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:13.671770    2576 patch_prober.go:28] interesting pod/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 14 08:03:13.671923 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:13.671839    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 14 08:03:13.736236 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:13.735415    2576 patch_prober.go:28] interesting pod/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 08:03:13.736236 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:13.735500    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 08:03:13.807233 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:13.806305    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl" event={"ID":"1b3d784e-2474-4eee-8f41-55de57b117f2","Type":"ContainerStarted","Data":"e6c62b3436407127df5d05e068fa37a245c97168a6866cfbe43fc156c0e73ced"}
Sep 14 08:03:13.807233 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:13.806366    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl" event={"ID":"1b3d784e-2474-4eee-8f41-55de57b117f2","Type":"ContainerStarted","Data":"70434b5a30c53c3d9d43aa4983469794373f570c796fc35f0044cdd6c259aaf3"}
Sep 14 08:03:13.807955 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:13.807923    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 14 08:03:13.808046 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:13.807980    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 14 08:03:13.825856 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:13.825782    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl" podStartSLOduration=1.82576019 podStartE2EDuration="1.82576019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:13.822601969 +0000 UTC m=+848.527140472" watchObservedRunningTime="2026-09-14 08:03:13.82576019 +0000 UTC m=+848.530298694"
Sep 14 08:03:13.831892 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:13.831858    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:13.978027 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:13.977933    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:14.084062 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:14.084025    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:14.236534 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:14.236435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:14.473417 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:14.471188    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:14.478747 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:14.478715    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:14.672278 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:14.672234    2576 patch_prober.go:28] interesting pod/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 14 08:03:14.672774 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:14.672314    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 14 08:03:14.735487 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:14.735444    2576 patch_prober.go:28] interesting pod/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 08:03:14.735681 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:14.735523    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 08:03:14.803811 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:14.803777    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 14 08:03:14.804088 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:14.804041    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 14 08:03:14.813991 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:14.813833    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:14.819061 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:14.819034    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:14.941298 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:14.941093    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:15.459021 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:15.458973    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:15.671875 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:15.671839    2576 patch_prober.go:28] interesting pod/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 14 08:03:15.672201 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:15.672167    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 14 08:03:15.735880 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:15.735790    2576 patch_prober.go:28] interesting pod/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 08:03:15.736473 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:15.736439    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 08:03:15.803462 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:15.803425    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 14 08:03:15.803756 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:15.803726    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 14 08:03:15.884091 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:15.884032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:15.988290 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:15.988112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:16.606043 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:16.605962    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:16.671856 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:16.671403    2576 patch_prober.go:28] interesting pod/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 14 08:03:16.671856 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:16.671482    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 14 08:03:16.735793 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:16.735751    2576 patch_prober.go:28] interesting pod/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 08:03:16.735962 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:16.735823    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 08:03:16.804013 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:16.803968    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 14 08:03:16.804235 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:16.804035    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 14 08:03:17.204097 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:17.204055    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:17.672004 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:17.671967    2576 patch_prober.go:28] interesting pod/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 14 08:03:17.672297 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:17.672262    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 14 08:03:17.736352 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:17.736309    2576 patch_prober.go:28] interesting pod/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 08:03:17.736863 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:17.736392    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 08:03:17.804253 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:17.804185    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 14 08:03:17.804432 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:17.804283    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 14 08:03:17.861687 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:17.861653    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:18.528820 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:18.528736    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:18.672656 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:18.672175    2576 patch_prober.go:28] interesting pod/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 14 08:03:18.672656 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:18.672267    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 14 08:03:18.735680 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:18.735200    2576 patch_prober.go:28] interesting pod/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 08:03:18.735680 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:18.735289    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 08:03:18.804278 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:18.804150    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 14 08:03:18.804278 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:18.804244    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 14 08:03:19.298880 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:19.298831    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:19.675553 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:19.675525    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:03:19.675836 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:19.675798    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:03:19.676508 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:19.676485    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:03:19.742232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:19.741401    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:03:19.742232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:19.741804    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:03:19.742743 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:19.742720    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:03:19.808975 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:19.808946    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:03:19.809419 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:19.809353    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:03:19.809994 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:19.809975    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:03:20.008708 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.008625    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"]
Sep 14 08:03:20.013430 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.013406    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.015535 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.015514    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-f67e60-serving-cert\""
Sep 14 08:03:20.023297 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.021427    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"]
Sep 14 08:03:20.039983 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.039962    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:20.118832 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.118805    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgbsn\" (UniqueName: \"kubernetes.io/projected/7950abdb-1459-4819-9165-545bb8752895-kube-api-access-hgbsn\") pod \"maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.118977 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.118847    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-maas-api-tls\") pod \"maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.118977 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.118872    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7950abdb-1459-4819-9165-545bb8752895-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.118977 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.118926    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-metrics-certs\") pod \"maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.140734 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.140707    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:20.219857 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.219824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgbsn\" (UniqueName: \"kubernetes.io/projected/7950abdb-1459-4819-9165-545bb8752895-kube-api-access-hgbsn\") pod \"maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.219857 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.219869    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-maas-api-tls\") pod \"maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.220117 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.219895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7950abdb-1459-4819-9165-545bb8752895-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.220117 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.219928    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-metrics-certs\") pod \"maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.220117 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:03:20.219998    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-serving-cert: secret "maas-api-e2e-inf-b-f67e60-serving-cert" not found
Sep 14 08:03:20.220117 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:03:20.220069    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-maas-api-tls podName:7950abdb-1459-4819-9165-545bb8752895 nodeName:}" failed. No retries permitted until 2026-09-14 08:03:20.720050401 +0000 UTC m=+855.424588895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-maas-api-tls") pod "maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr" (UID: "7950abdb-1459-4819-9165-545bb8752895") : secret "maas-api-e2e-inf-b-f67e60-serving-cert" not found
Sep 14 08:03:20.220870 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.220822    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7950abdb-1459-4819-9165-545bb8752895-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.223588 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.223541    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-metrics-certs\") pod \"maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.228756 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.228700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgbsn\" (UniqueName: \"kubernetes.io/projected/7950abdb-1459-4819-9165-545bb8752895-kube-api-access-hgbsn\") pod \"maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.318585 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.318502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:20.418819 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.418783    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:20.520740 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.520708    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:20.588562 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.588486    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:20.753234 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.752465    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-maas-api-tls\") pod \"maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.755602 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.755572    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-maas-api-tls\") pod \"maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.928742 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.928700    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:20.951098 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:20.951068    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:21.152412 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.152385    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:21.218504 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.218477    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"]
Sep 14 08:03:21.220028 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:03:21.219991    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7950abdb_1459_4819_9165_545bb8752895.slice/crio-48539be10666df17405a420f13618ac4c0902d6ba758e5c0108b4bcc94fcf813 WatchSource:0}: Error finding container 48539be10666df17405a420f13618ac4c0902d6ba758e5c0108b4bcc94fcf813: Status 404 returned error can't find the container with id 48539be10666df17405a420f13618ac4c0902d6ba758e5c0108b4bcc94fcf813
Sep 14 08:03:21.227727 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.227563    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:21.329175 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.329137    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:21.431801 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.431632    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:21.526545 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.526455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:21.840769 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.840655    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:21.867535 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.867499    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr" event={"ID":"7950abdb-1459-4819-9165-545bb8752895","Type":"ContainerStarted","Data":"6f783d4143101ce07b4317cb2b0a4adce59f05aefd2f8dd776e134e8714dc236"}
Sep 14 08:03:21.867727 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.867672    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:21.867800 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.867724    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr" event={"ID":"7950abdb-1459-4819-9165-545bb8752895","Type":"ContainerStarted","Data":"48539be10666df17405a420f13618ac4c0902d6ba758e5c0108b4bcc94fcf813"}
Sep 14 08:03:21.869532 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.869505    2576 generic.go:356] "Generic (PLEG): container finished" podID="8f528660-338e-4365-a5f8-c8aafef9bf5d" containerID="d9b85bfabd2a6a8233d44fc1e8395738f58cbf667aa6434e464fa878f33d7ec8" exitCode=0
Sep 14 08:03:21.869666 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.869578    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh" event={"ID":"8f528660-338e-4365-a5f8-c8aafef9bf5d","Type":"ContainerDied","Data":"d9b85bfabd2a6a8233d44fc1e8395738f58cbf667aa6434e464fa878f33d7ec8"}
Sep 14 08:03:21.869924 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.869905    2576 scope.go:117] "RemoveContainer" containerID="d9b85bfabd2a6a8233d44fc1e8395738f58cbf667aa6434e464fa878f33d7ec8"
Sep 14 08:03:21.871307 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.871277    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:21.871413 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.871346    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:21.871413 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.871395    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:21.885654 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:21.885608    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr" podStartSLOduration=2.467307872 podStartE2EDuration="2.885592289s" totalImagesPullingTime="418.284417ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:21.885048776 +0000 UTC m=+856.589587308" watchObservedRunningTime="2026-09-14 08:03:21.885592289 +0000 UTC m=+856.590130792"
Sep 14 08:03:22.625888 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:22.625744    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:22.880266 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:22.880133    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh" event={"ID":"8f528660-338e-4365-a5f8-c8aafef9bf5d","Type":"ContainerStarted","Data":"a86612389419a26839bd303acb35e6794eb550f3d0bba5b42bded8c383d6115d"}
Sep 14 08:03:23.284274 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:23.284165    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:23.304175 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:23.304146    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:03:23.304370 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:23.304340    2576 patch_prober.go:28] interesting pod/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 14 08:03:23.304448 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:23.304397    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh" podUID="8f528660-338e-4365-a5f8-c8aafef9bf5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 14 08:03:24.040320 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:24.040279    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:24.303590 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:24.303500    2576 patch_prober.go:28] interesting pod/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 14 08:03:24.303590 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:24.303555    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh" podUID="8f528660-338e-4365-a5f8-c8aafef9bf5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 14 08:03:24.480555 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:24.480514    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:24.708923 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:24.708884    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:25.303850 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:25.303816    2576 patch_prober.go:28] interesting pod/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 14 08:03:25.304258 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:25.303868    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh" podUID="8f528660-338e-4365-a5f8-c8aafef9bf5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 14 08:03:25.549116 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:25.549069    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:25.894046 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:25.894004    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:03:26.246032 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:26.245989    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:26.303705 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:26.303677    2576 patch_prober.go:28] interesting pod/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 14 08:03:26.303849 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:26.303728    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh" podUID="8f528660-338e-4365-a5f8-c8aafef9bf5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 14 08:03:26.908421 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:26.908381    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:27.304542 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:27.304457    2576 patch_prober.go:28] interesting pod/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 14 08:03:27.304542 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:27.304512    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh" podUID="8f528660-338e-4365-a5f8-c8aafef9bf5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 14 08:03:27.564136 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:27.563980    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:27.625585 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:27.625481    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:27.888625 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:27.888593    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:03:28.308291 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:28.308189    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:03:28.308627 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:28.308503    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:03:28.309287 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:28.309268    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:03:28.349519 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:28.349449    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:29.945229 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:29.944322    2576 generic.go:356] "Generic (PLEG): container finished" podID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerID="0e4f52e31112dc04b0253886a81fe9f99668226aa49724718574635562ec666c" exitCode=0
Sep 14 08:03:29.945229 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:29.944377    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" event={"ID":"271d49fc-4f91-466f-a1a3-800284dfa5d7","Type":"ContainerDied","Data":"0e4f52e31112dc04b0253886a81fe9f99668226aa49724718574635562ec666c"}
Sep 14 08:03:29.945229 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:29.945152    2576 scope.go:117] "RemoveContainer" containerID="0e4f52e31112dc04b0253886a81fe9f99668226aa49724718574635562ec666c"
Sep 14 08:03:29.946669 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:29.946629    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:29.946768 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:29.946721    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:29.946837 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:29.946767    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:30.951615 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:30.951580    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" event={"ID":"271d49fc-4f91-466f-a1a3-800284dfa5d7","Type":"ContainerStarted","Data":"12dfaeb69b1a57fd173c5c9ba758b669c33105a1b71db29a859f9a4618b809eb"}
Sep 14 08:03:31.461020 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:31.460975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:31.671106 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:31.671067    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:31.672127 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:31.672089    2576 patch_prober.go:28] interesting pod/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 14 08:03:31.672297 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:31.672153    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 14 08:03:32.003786 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:32.003738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:32.084353 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:32.080022    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:32.156241 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:32.138146    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:32.240240 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:32.239793    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:32.327278 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:32.326865    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"]
Sep 14 08:03:32.327278 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:32.327098    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh" podUID="8f528660-338e-4365-a5f8-c8aafef9bf5d" containerName="istio-proxy" containerID="cri-o://a86612389419a26839bd303acb35e6794eb550f3d0bba5b42bded8c383d6115d" gracePeriod=30
Sep 14 08:03:32.360232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:32.356893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:32.642394 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:32.642356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:32.670369 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:32.670344    2576 patch_prober.go:28] interesting pod/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 14 08:03:32.670513 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:32.670396    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 14 08:03:32.828233 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:32.826596    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:32.925829 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:32.925729    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:33.056963 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:33.056933    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:33.449681 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:33.449603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:33.653919 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:33.653877    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:33.670307 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:33.670272    2576 patch_prober.go:28] interesting pod/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 14 08:03:33.670439 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:33.670327    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 14 08:03:33.756202 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:33.756072    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:34.313871 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:34.313840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:34.613515 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:34.613489    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:34.670490 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:34.670457    2576 patch_prober.go:28] interesting pod/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 14 08:03:34.670636 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:34.670505    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 14 08:03:35.049375 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:35.049101    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:35.670615 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:35.670580    2576 patch_prober.go:28] interesting pod/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 14 08:03:35.671365 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:35.671329    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 14 08:03:35.735074 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:35.734996    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:36.392571 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:36.392537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:36.670777 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:36.670733    2576 patch_prober.go:28] interesting pod/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 14 08:03:36.671170 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:36.670800    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 14 08:03:36.990534 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:36.990458    2576 generic.go:356] "Generic (PLEG): container finished" podID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerID="e6c62b3436407127df5d05e068fa37a245c97168a6866cfbe43fc156c0e73ced" exitCode=0
Sep 14 08:03:36.990667 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:36.990536    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl" event={"ID":"1b3d784e-2474-4eee-8f41-55de57b117f2","Type":"ContainerDied","Data":"e6c62b3436407127df5d05e068fa37a245c97168a6866cfbe43fc156c0e73ced"}
Sep 14 08:03:36.990971 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:36.990954    2576 scope.go:117] "RemoveContainer" containerID="e6c62b3436407127df5d05e068fa37a245c97168a6866cfbe43fc156c0e73ced"
Sep 14 08:03:36.992217 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:36.992173    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:36.992310 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:36.992290    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:36.992356 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:36.992341    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:03:37.037124 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.037049    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:37.674055 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.674026    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:37.675291 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.675267    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:03:37.792711 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.792689    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:03:37.835547 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.835507    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:37.859843 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.859774    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-workload-certs\") pod \"8f528660-338e-4365-a5f8-c8aafef9bf5d\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") "
Sep 14 08:03:37.859843 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.859817    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-envoy\") pod \"8f528660-338e-4365-a5f8-c8aafef9bf5d\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") "
Sep 14 08:03:37.860021 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.859872    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-podinfo\") pod \"8f528660-338e-4365-a5f8-c8aafef9bf5d\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") "
Sep 14 08:03:37.860021 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.859904    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxlcs\" (UniqueName: \"kubernetes.io/projected/8f528660-338e-4365-a5f8-c8aafef9bf5d-kube-api-access-xxlcs\") pod \"8f528660-338e-4365-a5f8-c8aafef9bf5d\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") "
Sep 14 08:03:37.860021 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.859938    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-token\") pod \"8f528660-338e-4365-a5f8-c8aafef9bf5d\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") "
Sep 14 08:03:37.860021 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.859964    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-workload-socket\") pod \"8f528660-338e-4365-a5f8-c8aafef9bf5d\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") "
Sep 14 08:03:37.860259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.860021    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-credential-socket\") pod \"8f528660-338e-4365-a5f8-c8aafef9bf5d\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") "
Sep 14 08:03:37.860259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.860031    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f528660-338e-4365-a5f8-c8aafef9bf5d" (UID: "8f528660-338e-4365-a5f8-c8aafef9bf5d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:03:37.860259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.860066    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-data\") pod \"8f528660-338e-4365-a5f8-c8aafef9bf5d\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") "
Sep 14 08:03:37.860259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.860192    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f528660-338e-4365-a5f8-c8aafef9bf5d-istiod-ca-cert\") pod \"8f528660-338e-4365-a5f8-c8aafef9bf5d\" (UID: \"8f528660-338e-4365-a5f8-c8aafef9bf5d\") "
Sep 14 08:03:37.860469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.860297    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f528660-338e-4365-a5f8-c8aafef9bf5d" (UID: "8f528660-338e-4365-a5f8-c8aafef9bf5d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:03:37.860530 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.860477    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f528660-338e-4365-a5f8-c8aafef9bf5d" (UID: "8f528660-338e-4365-a5f8-c8aafef9bf5d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:03:37.860653 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.860615    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:03:37.860653 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.860640    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:03:37.860763 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.860655    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:03:37.860763 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.860667    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f528660-338e-4365-a5f8-c8aafef9bf5d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f528660-338e-4365-a5f8-c8aafef9bf5d" (UID: "8f528660-338e-4365-a5f8-c8aafef9bf5d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:37.860763 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.860686    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f528660-338e-4365-a5f8-c8aafef9bf5d" (UID: "8f528660-338e-4365-a5f8-c8aafef9bf5d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:03:37.862503 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.862482    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f528660-338e-4365-a5f8-c8aafef9bf5d" (UID: "8f528660-338e-4365-a5f8-c8aafef9bf5d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:03:37.862591 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.862516    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f528660-338e-4365-a5f8-c8aafef9bf5d-kube-api-access-xxlcs" (OuterVolumeSpecName: "kube-api-access-xxlcs") pod "8f528660-338e-4365-a5f8-c8aafef9bf5d" (UID: "8f528660-338e-4365-a5f8-c8aafef9bf5d"). InnerVolumeSpecName "kube-api-access-xxlcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:37.862634 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.862617    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f528660-338e-4365-a5f8-c8aafef9bf5d" (UID: "8f528660-338e-4365-a5f8-c8aafef9bf5d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:03:37.862674 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.862626    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f528660-338e-4365-a5f8-c8aafef9bf5d" (UID: "8f528660-338e-4365-a5f8-c8aafef9bf5d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:37.890445 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.890423    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:03:37.962157 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.962119    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f528660-338e-4365-a5f8-c8aafef9bf5d-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:03:37.962157 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.962148    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:03:37.962157 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.962160    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:03:37.962428 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.962171    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxlcs\" (UniqueName: \"kubernetes.io/projected/8f528660-338e-4365-a5f8-c8aafef9bf5d-kube-api-access-xxlcs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:03:37.962428 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.962185    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f528660-338e-4365-a5f8-c8aafef9bf5d-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:03:37.962428 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.962197    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f528660-338e-4365-a5f8-c8aafef9bf5d-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:03:37.999820 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:37.999780    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl" event={"ID":"1b3d784e-2474-4eee-8f41-55de57b117f2","Type":"ContainerStarted","Data":"df7927900110ea48cf014623e0684bdd75b5c5bffb35df42f83bad7b6d39c0ac"}
Sep 14 08:03:38.001504 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.001478    2576 generic.go:356] "Generic (PLEG): container finished" podID="8f528660-338e-4365-a5f8-c8aafef9bf5d" containerID="a86612389419a26839bd303acb35e6794eb550f3d0bba5b42bded8c383d6115d" exitCode=0
Sep 14 08:03:38.001617 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.001521    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"
Sep 14 08:03:38.001617 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.001559    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh" event={"ID":"8f528660-338e-4365-a5f8-c8aafef9bf5d","Type":"ContainerDied","Data":"a86612389419a26839bd303acb35e6794eb550f3d0bba5b42bded8c383d6115d"}
Sep 14 08:03:38.001617 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.001609    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh" event={"ID":"8f528660-338e-4365-a5f8-c8aafef9bf5d","Type":"ContainerDied","Data":"087ffd75d70ff96272bd89ff5d10744eb8ae63e1f297c98721a9ba69b78ad4f3"}
Sep 14 08:03:38.001746 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.001633    2576 scope.go:117] "RemoveContainer" containerID="a86612389419a26839bd303acb35e6794eb550f3d0bba5b42bded8c383d6115d"
Sep 14 08:03:38.011152 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.011135    2576 scope.go:117] "RemoveContainer" containerID="d9b85bfabd2a6a8233d44fc1e8395738f58cbf667aa6434e464fa878f33d7ec8"
Sep 14 08:03:38.019333 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.019312    2576 scope.go:117] "RemoveContainer" containerID="a86612389419a26839bd303acb35e6794eb550f3d0bba5b42bded8c383d6115d"
Sep 14 08:03:38.019597 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:03:38.019579    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a86612389419a26839bd303acb35e6794eb550f3d0bba5b42bded8c383d6115d\": container with ID starting with a86612389419a26839bd303acb35e6794eb550f3d0bba5b42bded8c383d6115d not found: ID does not exist" containerID="a86612389419a26839bd303acb35e6794eb550f3d0bba5b42bded8c383d6115d"
Sep 14 08:03:38.019659 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.019604    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a86612389419a26839bd303acb35e6794eb550f3d0bba5b42bded8c383d6115d"} err="failed to get container status \"a86612389419a26839bd303acb35e6794eb550f3d0bba5b42bded8c383d6115d\": rpc error: code = NotFound desc = could not find container \"a86612389419a26839bd303acb35e6794eb550f3d0bba5b42bded8c383d6115d\": container with ID starting with a86612389419a26839bd303acb35e6794eb550f3d0bba5b42bded8c383d6115d not found: ID does not exist"
Sep 14 08:03:38.019659 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.019620    2576 scope.go:117] "RemoveContainer" containerID="d9b85bfabd2a6a8233d44fc1e8395738f58cbf667aa6434e464fa878f33d7ec8"
Sep 14 08:03:38.019849 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:03:38.019835    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9b85bfabd2a6a8233d44fc1e8395738f58cbf667aa6434e464fa878f33d7ec8\": container with ID starting with d9b85bfabd2a6a8233d44fc1e8395738f58cbf667aa6434e464fa878f33d7ec8 not found: ID does not exist" containerID="d9b85bfabd2a6a8233d44fc1e8395738f58cbf667aa6434e464fa878f33d7ec8"
Sep 14 08:03:38.019888 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.019852    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9b85bfabd2a6a8233d44fc1e8395738f58cbf667aa6434e464fa878f33d7ec8"} err="failed to get container status \"d9b85bfabd2a6a8233d44fc1e8395738f58cbf667aa6434e464fa878f33d7ec8\": rpc error: code = NotFound desc = could not find container \"d9b85bfabd2a6a8233d44fc1e8395738f58cbf667aa6434e464fa878f33d7ec8\": container with ID starting with d9b85bfabd2a6a8233d44fc1e8395738f58cbf667aa6434e464fa878f33d7ec8 not found: ID does not exist"
Sep 14 08:03:38.043603 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.043555    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"]
Sep 14 08:03:38.047318 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.047287    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9ab35f8f-openshift-default-5784756fb8-nzggh"]
Sep 14 08:03:38.470893 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.470853    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:38.471106 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.471086    2576 patch_prober.go:28] interesting pod/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 08:03:38.471156 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.471135    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl" podUID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 08:03:38.620557 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.620514    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:38.726242 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:38.724118    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:39.464502 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:39.464466    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:39.471600 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:39.471573    2576 patch_prober.go:28] interesting pod/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 08:03:39.471730 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:39.471627    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl" podUID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 08:03:39.842590 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:39.842507    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f528660-338e-4365-a5f8-c8aafef9bf5d" path="/var/lib/kubelet/pods/8f528660-338e-4365-a5f8-c8aafef9bf5d/volumes"
Sep 14 08:03:40.326475 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:40.326331    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:40.471766 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:40.471729    2576 patch_prober.go:28] interesting pod/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 08:03:40.471930 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:40.471780    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl" podUID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 08:03:40.880595 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:40.880560    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:41.113238 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.113190    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"]
Sep 14 08:03:41.113709 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.113694    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f528660-338e-4365-a5f8-c8aafef9bf5d" containerName="istio-proxy"
Sep 14 08:03:41.113753 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.113711    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f528660-338e-4365-a5f8-c8aafef9bf5d" containerName="istio-proxy"
Sep 14 08:03:41.113753 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.113728    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f528660-338e-4365-a5f8-c8aafef9bf5d" containerName="istio-proxy"
Sep 14 08:03:41.113753 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.113734    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f528660-338e-4365-a5f8-c8aafef9bf5d" containerName="istio-proxy"
Sep 14 08:03:41.113850 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.113816    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f528660-338e-4365-a5f8-c8aafef9bf5d" containerName="istio-proxy"
Sep 14 08:03:41.113850 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.113824    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f528660-338e-4365-a5f8-c8aafef9bf5d" containerName="istio-proxy"
Sep 14 08:03:41.117075 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.117053    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:41.119295 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.119261    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-c3bb3b8f-serving-cert\""
Sep 14 08:03:41.143105 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.143073    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"]
Sep 14 08:03:41.150287 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.146617    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:41.193269 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.193248    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pkzb\" (UniqueName: \"kubernetes.io/projected/c8c250a7-490e-4434-8e6d-b4363d9b7948-kube-api-access-4pkzb\") pod \"maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:41.193445 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.193430    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8c250a7-490e-4434-8e6d-b4363d9b7948-metrics-certs\") pod \"maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:41.193549 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.193538    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c250a7-490e-4434-8e6d-b4363d9b7948-openshift-service-ca\") pod \"maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:41.193672 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.193663    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8c250a7-490e-4434-8e6d-b4363d9b7948-maas-api-tls\") pod \"maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:41.251142 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.251109    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:41.294648 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.294610    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8c250a7-490e-4434-8e6d-b4363d9b7948-maas-api-tls\") pod \"maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:41.294826 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.294704    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pkzb\" (UniqueName: \"kubernetes.io/projected/c8c250a7-490e-4434-8e6d-b4363d9b7948-kube-api-access-4pkzb\") pod \"maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:41.294826 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.294742    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8c250a7-490e-4434-8e6d-b4363d9b7948-metrics-certs\") pod \"maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:41.294826 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.294777    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c250a7-490e-4434-8e6d-b4363d9b7948-openshift-service-ca\") pod \"maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:41.295522 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.295491    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c250a7-490e-4434-8e6d-b4363d9b7948-openshift-service-ca\") pod \"maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:41.297162 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.297144    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8c250a7-490e-4434-8e6d-b4363d9b7948-maas-api-tls\") pod \"maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:41.297252 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.297167    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8c250a7-490e-4434-8e6d-b4363d9b7948-metrics-certs\") pod \"maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:41.301963 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.301942    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pkzb\" (UniqueName: \"kubernetes.io/projected/c8c250a7-490e-4434-8e6d-b4363d9b7948-kube-api-access-4pkzb\") pod \"maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:41.341476 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.341443    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:41.456411 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.456289    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:41.471865 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.471834    2576 patch_prober.go:28] interesting pod/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 08:03:41.471973 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.471914    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl" podUID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 08:03:41.603479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.603446    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:41.696091 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.696054    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:41.750297 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:41.743780    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"]
Sep 14 08:03:41.754549 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:03:41.754517    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8c250a7_490e_4434_8e6d_b4363d9b7948.slice/crio-2c7c1c5786b8631879acffbd03b8a3e277ef8f253df1bc238298d8c92ddd19ac WatchSource:0}: Error finding container 2c7c1c5786b8631879acffbd03b8a3e277ef8f253df1bc238298d8c92ddd19ac: Status 404 returned error can't find the container with id 2c7c1c5786b8631879acffbd03b8a3e277ef8f253df1bc238298d8c92ddd19ac
Sep 14 08:03:42.026025 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:42.025902    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw" event={"ID":"c8c250a7-490e-4434-8e6d-b4363d9b7948","Type":"ContainerStarted","Data":"2c7c1c5786b8631879acffbd03b8a3e277ef8f253df1bc238298d8c92ddd19ac"}
Sep 14 08:03:42.146908 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:42.146873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:42.471872 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:42.471842    2576 patch_prober.go:28] interesting pod/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 08:03:42.472029 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:42.471898    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl" podUID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 08:03:42.934411 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:42.934379    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:43.471948 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:43.471917    2576 patch_prober.go:28] interesting pod/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 08:03:43.472308 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:43.471971    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl" podUID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 08:03:43.554150 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:43.554069    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:44.040000 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:44.039967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw" event={"ID":"c8c250a7-490e-4434-8e6d-b4363d9b7948","Type":"ContainerStarted","Data":"69d8a070886cedb5dfc996bb3294a664087b12ab054979ac28dbbe00f787ad82"}
Sep 14 08:03:44.040142 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:44.040079    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:44.059394 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:44.059334    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw" podStartSLOduration=1.523218419 podStartE2EDuration="3.059314461s" totalImagesPullingTime="1.536096042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:44.054922113 +0000 UTC m=+878.759460616" watchObservedRunningTime="2026-09-14 08:03:44.059314461 +0000 UTC m=+878.763852965"
Sep 14 08:03:44.264827 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:44.264790    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:44.475661 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:44.475632    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:44.476114 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:44.476030    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:44.476675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:44.476657    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:03:50.050093 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:50.050064    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:03:52.390762 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:03:52.390727    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:04:02.952506 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:02.952472    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:03.721830 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:03.721796    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:04.443354 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:04.443323    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:05.133126 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:05.133092    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:05.845932 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:05.845906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:06.649282 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:06.649234    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:07.268192 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:07.268140    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:08.013232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:08.013181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:08.876419 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:08.876379    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:09.285102 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:09.285002    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:09.795487 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:09.795450    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:10.411763 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.411728    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"]
Sep 14 08:04:10.412231 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.412057    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw" podUID="c8c250a7-490e-4434-8e6d-b4363d9b7948" containerName="maas-api" containerID="cri-o://69d8a070886cedb5dfc996bb3294a664087b12ab054979ac28dbbe00f787ad82" gracePeriod=30
Sep 14 08:04:10.441472 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.441441    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:10.511005 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.510967    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:10.575846 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.575817    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:10.659994 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.659970    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:04:10.667615 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.667562    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8c250a7-490e-4434-8e6d-b4363d9b7948-metrics-certs\") pod \"c8c250a7-490e-4434-8e6d-b4363d9b7948\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") "
Sep 14 08:04:10.667710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.667631    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pkzb\" (UniqueName: \"kubernetes.io/projected/c8c250a7-490e-4434-8e6d-b4363d9b7948-kube-api-access-4pkzb\") pod \"c8c250a7-490e-4434-8e6d-b4363d9b7948\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") "
Sep 14 08:04:10.667710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.667659    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c250a7-490e-4434-8e6d-b4363d9b7948-openshift-service-ca\") pod \"c8c250a7-490e-4434-8e6d-b4363d9b7948\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") "
Sep 14 08:04:10.667710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.667675    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8c250a7-490e-4434-8e6d-b4363d9b7948-maas-api-tls\") pod \"c8c250a7-490e-4434-8e6d-b4363d9b7948\" (UID: \"c8c250a7-490e-4434-8e6d-b4363d9b7948\") "
Sep 14 08:04:10.668049 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.668020    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8c250a7-490e-4434-8e6d-b4363d9b7948-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c8c250a7-490e-4434-8e6d-b4363d9b7948" (UID: "c8c250a7-490e-4434-8e6d-b4363d9b7948"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:04:10.669781 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.669749    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8c250a7-490e-4434-8e6d-b4363d9b7948-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c8c250a7-490e-4434-8e6d-b4363d9b7948" (UID: "c8c250a7-490e-4434-8e6d-b4363d9b7948"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:04:10.669872 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.669861    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8c250a7-490e-4434-8e6d-b4363d9b7948-kube-api-access-4pkzb" (OuterVolumeSpecName: "kube-api-access-4pkzb") pod "c8c250a7-490e-4434-8e6d-b4363d9b7948" (UID: "c8c250a7-490e-4434-8e6d-b4363d9b7948"). InnerVolumeSpecName "kube-api-access-4pkzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:10.669925 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.669865    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8c250a7-490e-4434-8e6d-b4363d9b7948-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c8c250a7-490e-4434-8e6d-b4363d9b7948" (UID: "c8c250a7-490e-4434-8e6d-b4363d9b7948"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:04:10.673368 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.673345    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:10.768402 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.768373    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8c250a7-490e-4434-8e6d-b4363d9b7948-metrics-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:04:10.768402 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.768399    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pkzb\" (UniqueName: \"kubernetes.io/projected/c8c250a7-490e-4434-8e6d-b4363d9b7948-kube-api-access-4pkzb\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:04:10.768574 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.768412    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c250a7-490e-4434-8e6d-b4363d9b7948-openshift-service-ca\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:04:10.768574 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.768421    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8c250a7-490e-4434-8e6d-b4363d9b7948-maas-api-tls\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:04:10.773980 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:10.773956    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:11.097530 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.097416    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:11.178272 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.178242    2576 generic.go:356] "Generic (PLEG): container finished" podID="c8c250a7-490e-4434-8e6d-b4363d9b7948" containerID="69d8a070886cedb5dfc996bb3294a664087b12ab054979ac28dbbe00f787ad82" exitCode=0
Sep 14 08:04:11.178427 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.178288    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"
Sep 14 08:04:11.178427 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.178322    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw" event={"ID":"c8c250a7-490e-4434-8e6d-b4363d9b7948","Type":"ContainerDied","Data":"69d8a070886cedb5dfc996bb3294a664087b12ab054979ac28dbbe00f787ad82"}
Sep 14 08:04:11.178427 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.178372    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw" event={"ID":"c8c250a7-490e-4434-8e6d-b4363d9b7948","Type":"ContainerDied","Data":"2c7c1c5786b8631879acffbd03b8a3e277ef8f253df1bc238298d8c92ddd19ac"}
Sep 14 08:04:11.178427 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.178396    2576 scope.go:117] "RemoveContainer" containerID="69d8a070886cedb5dfc996bb3294a664087b12ab054979ac28dbbe00f787ad82"
Sep 14 08:04:11.188036 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.188022    2576 scope.go:117] "RemoveContainer" containerID="69d8a070886cedb5dfc996bb3294a664087b12ab054979ac28dbbe00f787ad82"
Sep 14 08:04:11.188336 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:04:11.188314    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69d8a070886cedb5dfc996bb3294a664087b12ab054979ac28dbbe00f787ad82\": container with ID starting with 69d8a070886cedb5dfc996bb3294a664087b12ab054979ac28dbbe00f787ad82 not found: ID does not exist" containerID="69d8a070886cedb5dfc996bb3294a664087b12ab054979ac28dbbe00f787ad82"
Sep 14 08:04:11.188437 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.188343    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69d8a070886cedb5dfc996bb3294a664087b12ab054979ac28dbbe00f787ad82"} err="failed to get container status \"69d8a070886cedb5dfc996bb3294a664087b12ab054979ac28dbbe00f787ad82\": rpc error: code = NotFound desc = could not find container \"69d8a070886cedb5dfc996bb3294a664087b12ab054979ac28dbbe00f787ad82\": container with ID starting with 69d8a070886cedb5dfc996bb3294a664087b12ab054979ac28dbbe00f787ad82 not found: ID does not exist"
Sep 14 08:04:11.201395 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.201372    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"]
Sep 14 08:04:11.203438 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.203419    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c3bb3b8f-5df78dbc96-jwpmw"]
Sep 14 08:04:11.242430 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.242406    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:11.326100 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.326070    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:11.569046 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.569011    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"]
Sep 14 08:04:11.569691 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.569670    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8c250a7-490e-4434-8e6d-b4363d9b7948" containerName="maas-api"
Sep 14 08:04:11.569736 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.569697    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8c250a7-490e-4434-8e6d-b4363d9b7948" containerName="maas-api"
Sep 14 08:04:11.569829 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.569817    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8c250a7-490e-4434-8e6d-b4363d9b7948" containerName="maas-api"
Sep 14 08:04:11.573323 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.573294    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.575345 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.575299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-workload-certs\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.575345 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.575355    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-credential-socket\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.575546 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.575413    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-podinfo\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.575546 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.575476    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-envoy\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.575546 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.575510    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-workload-socket\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.575715 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.575598    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5k7b\" (UniqueName: \"kubernetes.io/projected/40d7fe21-740a-4a9a-b98a-3f0998363af9-kube-api-access-v5k7b\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.575715 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.575685    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ddcb78c8-openshift-default-dockercfg-sgnwc\""
Sep 14 08:04:11.575715 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.575690    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-data\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.575876 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.575721    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-token\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.575876 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.575754    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40d7fe21-740a-4a9a-b98a-3f0998363af9-istiod-ca-cert\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.581946 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.581922    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"]
Sep 14 08:04:11.596885 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.596857    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:11.676812 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.676780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-envoy\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.676995 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.676827    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-workload-socket\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.676995 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.676870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5k7b\" (UniqueName: \"kubernetes.io/projected/40d7fe21-740a-4a9a-b98a-3f0998363af9-kube-api-access-v5k7b\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.676995 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.676919    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-data\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.676995 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.676939    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-token\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.676995 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.676969    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40d7fe21-740a-4a9a-b98a-3f0998363af9-istiod-ca-cert\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.677282 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.677025    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-workload-certs\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.677282 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.677056    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-credential-socket\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.677282 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.677097    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-podinfo\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.677451 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.677351    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-workload-socket\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.677588 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.677560    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-workload-certs\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.677903 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.677878    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-data\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.677903 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.677887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40d7fe21-740a-4a9a-b98a-3f0998363af9-istiod-ca-cert\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.678083 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.677999    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-credential-socket\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.679351 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.679330    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-envoy\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.679670 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.679651    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-podinfo\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.684079 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.684057    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:11.684828 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.684807    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-token\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.684972 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.684953    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5k7b\" (UniqueName: \"kubernetes.io/projected/40d7fe21-740a-4a9a-b98a-3f0998363af9-kube-api-access-v5k7b\") pod \"e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") " pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:11.839465 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.839391    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8c250a7-490e-4434-8e6d-b4363d9b7948" path="/var/lib/kubelet/pods/c8c250a7-490e-4434-8e6d-b4363d9b7948/volumes"
Sep 14 08:04:11.845538 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.845516    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:11.885274 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:11.885252    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:12.168769 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:12.168742    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"]
Sep 14 08:04:12.170559 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:04:12.170535    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40d7fe21_740a_4a9a_b98a_3f0998363af9.slice/crio-d68e827e1c76268f4fd1dfa2a402e7a9f487a177799af86423108b9f49a7e7e5 WatchSource:0}: Error finding container d68e827e1c76268f4fd1dfa2a402e7a9f487a177799af86423108b9f49a7e7e5: Status 404 returned error can't find the container with id d68e827e1c76268f4fd1dfa2a402e7a9f487a177799af86423108b9f49a7e7e5
Sep 14 08:04:12.172961 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:12.172929    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:04:12.173027 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:12.173008    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:04:12.173071 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:12.173056    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:04:12.185654 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:12.185634    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf" event={"ID":"40d7fe21-740a-4a9a-b98a-3f0998363af9","Type":"ContainerStarted","Data":"d68e827e1c76268f4fd1dfa2a402e7a9f487a177799af86423108b9f49a7e7e5"}
Sep 14 08:04:12.517565 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:12.517477    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:12.620130 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:12.620089    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:04:13.192809 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:13.192773    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf" event={"ID":"40d7fe21-740a-4a9a-b98a-3f0998363af9","Type":"ContainerStarted","Data":"e7dd7c5ba7c607ca09abb16cfcf6a6aef8d3bf296510e20d5115bee054a40126"}
Sep 14 08:04:13.214317 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:13.214267    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf" podStartSLOduration=2.214251785 podStartE2EDuration="2.214251785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:13.213052603 +0000 UTC m=+907.917591123" watchObservedRunningTime="2026-09-14 08:04:13.214251785 +0000 UTC m=+907.918790286"
Sep 14 08:04:13.294654 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:13.294628    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:13.886087 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:13.886052    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:13.891414 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:13.891387    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:14.126262 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:14.126225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:14.203014 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:14.202922    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:14.206345 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:14.206317    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:15.786940 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:15.786900    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:15.888333 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:15.888300    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:16.432187 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:16.432157    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:17.094611 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:17.094575    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:17.742431 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:17.742399    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:18.534330 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:18.534303    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:18.585097 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:18.585064    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:19.065038 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:19.065006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:19.168606 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:19.168573    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:19.241851 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:19.241820    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:22.779911 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:22.779875    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:25.208111 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:25.208072    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:04:31.804489 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.804455    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"]
Sep 14 08:04:31.807639 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.807623    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.811033 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.811013    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-831f8e\"/\"simulator-w0-831f8e-kserve-self-signed-certs\""
Sep 14 08:04:31.811133 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.811062    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-831f8e\"/\"kube-root-ca.crt\""
Sep 14 08:04:31.811133 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.811094    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-831f8e\"/\"default-dockercfg-svws5\""
Sep 14 08:04:31.811133 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.811120    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-831f8e\"/\"openshift-service-ca.crt\""
Sep 14 08:04:31.815116 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.815092    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"]
Sep 14 08:04:31.885573 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.885535    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-home\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.885745 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.885584    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-tls-certs\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.885745 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.885625    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-dshm\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.885745 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.885722    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-model-cache\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.885921 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.885775    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm99g\" (UniqueName: \"kubernetes.io/projected/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-kube-api-access-wm99g\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.885921 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.885837    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-tmp-dir\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.911083 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.911060    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:31.987318 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.987278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-home\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.987318 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.987320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-tls-certs\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.987577 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.987343    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-dshm\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.987577 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.987466    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-model-cache\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.987577 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.987565    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm99g\" (UniqueName: \"kubernetes.io/projected/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-kube-api-access-wm99g\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.987737 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.987621    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-tmp-dir\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.987821 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.987792    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-home\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.987943 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.987901    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-model-cache\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.988084 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.987988    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-tmp-dir\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.990295 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.990267    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-dshm\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.991394 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.991371    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-tls-certs\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:31.995732 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:31.995713    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm99g\" (UniqueName: \"kubernetes.io/projected/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-kube-api-access-wm99g\") pod \"simulator-w0-831f8e-kserve-6d76875986-sq6z7\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") " pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:32.118911 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:32.118878    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:32.395083 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:32.395052    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"]
Sep 14 08:04:32.395922 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:04:32.395891    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26ff0142_c0d1_4d7e_80bc_7fc494fb7d0d.slice/crio-12adb80c3bbad2db41495d4528e88f6bf356dd6c215e2f8c70e25e234a867f53 WatchSource:0}: Error finding container 12adb80c3bbad2db41495d4528e88f6bf356dd6c215e2f8c70e25e234a867f53: Status 404 returned error can't find the container with id 12adb80c3bbad2db41495d4528e88f6bf356dd6c215e2f8c70e25e234a867f53
Sep 14 08:04:33.306676 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:33.306631    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7" event={"ID":"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d","Type":"ContainerStarted","Data":"6fe0893c707ae069357a2f13c440375af4ace228eabf182e75ce8643c940a893"}
Sep 14 08:04:33.307198 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:33.306688    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7" event={"ID":"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d","Type":"ContainerStarted","Data":"12adb80c3bbad2db41495d4528e88f6bf356dd6c215e2f8c70e25e234a867f53"}
Sep 14 08:04:33.323816 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:33.323759    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7" podStartSLOduration=2.323744629 podStartE2EDuration="2.323744629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:33.32114307 +0000 UTC m=+928.025681572" watchObservedRunningTime="2026-09-14 08:04:33.323744629 +0000 UTC m=+928.028283130"
Sep 14 08:04:35.321511 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:35.321484    2576 generic.go:356] "Generic (PLEG): container finished" podID="40d7fe21-740a-4a9a-b98a-3f0998363af9" containerID="e7dd7c5ba7c607ca09abb16cfcf6a6aef8d3bf296510e20d5115bee054a40126" exitCode=0
Sep 14 08:04:35.321880 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:35.321548    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf" event={"ID":"40d7fe21-740a-4a9a-b98a-3f0998363af9","Type":"ContainerDied","Data":"e7dd7c5ba7c607ca09abb16cfcf6a6aef8d3bf296510e20d5115bee054a40126"}
Sep 14 08:04:35.321950 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:35.321932    2576 scope.go:117] "RemoveContainer" containerID="e7dd7c5ba7c607ca09abb16cfcf6a6aef8d3bf296510e20d5115bee054a40126"
Sep 14 08:04:35.323045 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:35.323014    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:04:35.323102 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:35.323084    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:04:35.323145 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:35.323113    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:04:36.341850 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:36.341799    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf" event={"ID":"40d7fe21-740a-4a9a-b98a-3f0998363af9","Type":"ContainerStarted","Data":"ac4e6d7f18d79a84b1116a755ca300e3c65cc3789b9b8a600eff8326b69e0975"}
Sep 14 08:04:36.885856 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:36.885819    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:36.886057 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:36.886031    2576 patch_prober.go:28] interesting pod/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 14 08:04:36.886149 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:36.886093    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf" podUID="40d7fe21-740a-4a9a-b98a-3f0998363af9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 14 08:04:37.886553 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:37.886519    2576 patch_prober.go:28] interesting pod/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 14 08:04:37.886999 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:37.886587    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf" podUID="40d7fe21-740a-4a9a-b98a-3f0998363af9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 14 08:04:38.105128 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:38.105095    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:04:38.886552 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:38.886516    2576 patch_prober.go:28] interesting pod/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 14 08:04:38.886726 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:38.886573    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf" podUID="40d7fe21-740a-4a9a-b98a-3f0998363af9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 14 08:04:39.885561 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:39.885524    2576 patch_prober.go:28] interesting pod/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 14 08:04:39.885749 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:39.885591    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf" podUID="40d7fe21-740a-4a9a-b98a-3f0998363af9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 14 08:04:40.885839 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:40.885795    2576 patch_prober.go:28] interesting pod/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 14 08:04:40.886262 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:40.885857    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf" podUID="40d7fe21-740a-4a9a-b98a-3f0998363af9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 14 08:04:41.579935 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:41.579888    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:41.889780 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:41.889749    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:41.890246 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:41.890145    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:41.890759 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:41.890740    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:04:42.111021 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:42.110988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:42.119972 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:42.119949    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:42.120094 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:42.119988    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:42.132566 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:42.132543    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:42.387953 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:42.387919    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:04:42.752696 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:42.752611    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:43.383238 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:43.383168    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:44.145796 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:44.145768    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:44.786662 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:44.786630    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:45.512088 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:45.512050    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:46.252871 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:46.252841    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:46.863652 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:46.863620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:47.636593 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:47.636562    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:48.254131 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:48.254098    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:48.878845 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:48.878807    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:49.640319 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:49.640285    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:50.258249 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:50.258220    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:50.928486 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:50.928457    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:55.878603 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:55.878574    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:56.438259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:56.438198    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:56.503224 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:56.503182    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:56.615829 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:56.615795    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:56.721731 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:56.721653    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:57.926673 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:57.926639    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"]
Sep 14 08:04:57.927169 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:57.926927    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf" podUID="40d7fe21-740a-4a9a-b98a-3f0998363af9" containerName="istio-proxy" containerID="cri-o://ac4e6d7f18d79a84b1116a755ca300e3c65cc3789b9b8a600eff8326b69e0975" gracePeriod=30
Sep 14 08:04:58.242154 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:58.242080    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:58.344031 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:58.344003    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:58.445224 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:58.445182    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:58.543502 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:58.543394    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:58.731790 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:58.731748    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:58.992032 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:58.991995    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:59.272895 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:04:59.272815    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:00.229916 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:00.229884    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:00.869678 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:00.869645    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:00.976480 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:00.976449    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:01.070632 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:01.070599    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:01.849446 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:01.849413    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:02.573224 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:02.573188    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:03.174776 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.174752    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:05:03.287360 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.287295    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-workload-socket\") pod \"40d7fe21-740a-4a9a-b98a-3f0998363af9\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") "
Sep 14 08:05:03.287360 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.287326    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5k7b\" (UniqueName: \"kubernetes.io/projected/40d7fe21-740a-4a9a-b98a-3f0998363af9-kube-api-access-v5k7b\") pod \"40d7fe21-740a-4a9a-b98a-3f0998363af9\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") "
Sep 14 08:05:03.287360 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.287350    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-token\") pod \"40d7fe21-740a-4a9a-b98a-3f0998363af9\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") "
Sep 14 08:05:03.287582 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.287378    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40d7fe21-740a-4a9a-b98a-3f0998363af9-istiod-ca-cert\") pod \"40d7fe21-740a-4a9a-b98a-3f0998363af9\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") "
Sep 14 08:05:03.287582 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.287536    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-podinfo\") pod \"40d7fe21-740a-4a9a-b98a-3f0998363af9\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") "
Sep 14 08:05:03.287694 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.287581    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-workload-certs\") pod \"40d7fe21-740a-4a9a-b98a-3f0998363af9\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") "
Sep 14 08:05:03.287694 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.287593    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "40d7fe21-740a-4a9a-b98a-3f0998363af9" (UID: "40d7fe21-740a-4a9a-b98a-3f0998363af9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:05:03.287798 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.287714    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-data\") pod \"40d7fe21-740a-4a9a-b98a-3f0998363af9\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") "
Sep 14 08:05:03.287798 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.287770    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-envoy\") pod \"40d7fe21-740a-4a9a-b98a-3f0998363af9\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") "
Sep 14 08:05:03.287901 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.287794    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "40d7fe21-740a-4a9a-b98a-3f0998363af9" (UID: "40d7fe21-740a-4a9a-b98a-3f0998363af9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:05:03.287901 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.287819    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-credential-socket\") pod \"40d7fe21-740a-4a9a-b98a-3f0998363af9\" (UID: \"40d7fe21-740a-4a9a-b98a-3f0998363af9\") "
Sep 14 08:05:03.287901 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.287838    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40d7fe21-740a-4a9a-b98a-3f0998363af9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "40d7fe21-740a-4a9a-b98a-3f0998363af9" (UID: "40d7fe21-740a-4a9a-b98a-3f0998363af9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:05:03.288058 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.287944    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-data" (OuterVolumeSpecName: "istio-data") pod "40d7fe21-740a-4a9a-b98a-3f0998363af9" (UID: "40d7fe21-740a-4a9a-b98a-3f0998363af9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:05:03.288161 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.288143    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "40d7fe21-740a-4a9a-b98a-3f0998363af9" (UID: "40d7fe21-740a-4a9a-b98a-3f0998363af9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:05:03.288332 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.288312    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:05:03.288383 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.288340    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:05:03.288383 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.288356    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:05:03.288383 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.288371    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40d7fe21-740a-4a9a-b98a-3f0998363af9-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:05:03.288480 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.288384    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:05:03.289604 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.289585    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40d7fe21-740a-4a9a-b98a-3f0998363af9-kube-api-access-v5k7b" (OuterVolumeSpecName: "kube-api-access-v5k7b") pod "40d7fe21-740a-4a9a-b98a-3f0998363af9" (UID: "40d7fe21-740a-4a9a-b98a-3f0998363af9"). InnerVolumeSpecName "kube-api-access-v5k7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:05:03.289763 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.289747    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "40d7fe21-740a-4a9a-b98a-3f0998363af9" (UID: "40d7fe21-740a-4a9a-b98a-3f0998363af9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:05:03.289973 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.289949    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "40d7fe21-740a-4a9a-b98a-3f0998363af9" (UID: "40d7fe21-740a-4a9a-b98a-3f0998363af9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:05:03.290035 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.289977    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-token" (OuterVolumeSpecName: "istio-token") pod "40d7fe21-740a-4a9a-b98a-3f0998363af9" (UID: "40d7fe21-740a-4a9a-b98a-3f0998363af9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:05:03.346401 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.346377    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:03.389129 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.389105    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:05:03.389129 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.389128    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5k7b\" (UniqueName: \"kubernetes.io/projected/40d7fe21-740a-4a9a-b98a-3f0998363af9-kube-api-access-v5k7b\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:05:03.389289 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.389138    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:05:03.389289 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.389148    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40d7fe21-740a-4a9a-b98a-3f0998363af9-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:05:03.492789 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.492752    2576 generic.go:356] "Generic (PLEG): container finished" podID="40d7fe21-740a-4a9a-b98a-3f0998363af9" containerID="ac4e6d7f18d79a84b1116a755ca300e3c65cc3789b9b8a600eff8326b69e0975" exitCode=0
Sep 14 08:05:03.492958 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.492808    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"
Sep 14 08:05:03.492958 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.492833    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf" event={"ID":"40d7fe21-740a-4a9a-b98a-3f0998363af9","Type":"ContainerDied","Data":"ac4e6d7f18d79a84b1116a755ca300e3c65cc3789b9b8a600eff8326b69e0975"}
Sep 14 08:05:03.492958 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.492891    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf" event={"ID":"40d7fe21-740a-4a9a-b98a-3f0998363af9","Type":"ContainerDied","Data":"d68e827e1c76268f4fd1dfa2a402e7a9f487a177799af86423108b9f49a7e7e5"}
Sep 14 08:05:03.492958 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.492915    2576 scope.go:117] "RemoveContainer" containerID="ac4e6d7f18d79a84b1116a755ca300e3c65cc3789b9b8a600eff8326b69e0975"
Sep 14 08:05:03.497661 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.497635    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:05:03.503099 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.503084    2576 scope.go:117] "RemoveContainer" containerID="e7dd7c5ba7c607ca09abb16cfcf6a6aef8d3bf296510e20d5115bee054a40126"
Sep 14 08:05:03.510978 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.510961    2576 scope.go:117] "RemoveContainer" containerID="ac4e6d7f18d79a84b1116a755ca300e3c65cc3789b9b8a600eff8326b69e0975"
Sep 14 08:05:03.511228 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:05:03.511194    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac4e6d7f18d79a84b1116a755ca300e3c65cc3789b9b8a600eff8326b69e0975\": container with ID starting with ac4e6d7f18d79a84b1116a755ca300e3c65cc3789b9b8a600eff8326b69e0975 not found: ID does not exist" containerID="ac4e6d7f18d79a84b1116a755ca300e3c65cc3789b9b8a600eff8326b69e0975"
Sep 14 08:05:03.511271 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.511240    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac4e6d7f18d79a84b1116a755ca300e3c65cc3789b9b8a600eff8326b69e0975"} err="failed to get container status \"ac4e6d7f18d79a84b1116a755ca300e3c65cc3789b9b8a600eff8326b69e0975\": rpc error: code = NotFound desc = could not find container \"ac4e6d7f18d79a84b1116a755ca300e3c65cc3789b9b8a600eff8326b69e0975\": container with ID starting with ac4e6d7f18d79a84b1116a755ca300e3c65cc3789b9b8a600eff8326b69e0975 not found: ID does not exist"
Sep 14 08:05:03.511271 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.511265    2576 scope.go:117] "RemoveContainer" containerID="e7dd7c5ba7c607ca09abb16cfcf6a6aef8d3bf296510e20d5115bee054a40126"
Sep 14 08:05:03.511523 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:05:03.511507    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7dd7c5ba7c607ca09abb16cfcf6a6aef8d3bf296510e20d5115bee054a40126\": container with ID starting with e7dd7c5ba7c607ca09abb16cfcf6a6aef8d3bf296510e20d5115bee054a40126 not found: ID does not exist" containerID="e7dd7c5ba7c607ca09abb16cfcf6a6aef8d3bf296510e20d5115bee054a40126"
Sep 14 08:05:03.511561 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.511530    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7dd7c5ba7c607ca09abb16cfcf6a6aef8d3bf296510e20d5115bee054a40126"} err="failed to get container status \"e7dd7c5ba7c607ca09abb16cfcf6a6aef8d3bf296510e20d5115bee054a40126\": rpc error: code = NotFound desc = could not find container \"e7dd7c5ba7c607ca09abb16cfcf6a6aef8d3bf296510e20d5115bee054a40126\": container with ID starting with e7dd7c5ba7c607ca09abb16cfcf6a6aef8d3bf296510e20d5115bee054a40126 not found: ID does not exist"
Sep 14 08:05:03.514688 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.514667    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"]
Sep 14 08:05:03.520644 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.520621    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ddcb78c8-openshift-default-579b75c59d-xrbtf"]
Sep 14 08:05:03.839481 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:03.839446    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40d7fe21-740a-4a9a-b98a-3f0998363af9" path="/var/lib/kubelet/pods/40d7fe21-740a-4a9a-b98a-3f0998363af9/volumes"
Sep 14 08:05:04.110945 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:04.110921    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:04.754991 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:04.754960    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:05.538880 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:05.538849    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:05.641632 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:05.641193    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:06.000066 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:06.000036    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:06.085901 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:06.085819    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:06.192557 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:06.192523    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:11.169148 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.169109    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"]
Sep 14 08:05:11.169709 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.169689    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40d7fe21-740a-4a9a-b98a-3f0998363af9" containerName="istio-proxy"
Sep 14 08:05:11.169794 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.169711    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="40d7fe21-740a-4a9a-b98a-3f0998363af9" containerName="istio-proxy"
Sep 14 08:05:11.169794 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.169728    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40d7fe21-740a-4a9a-b98a-3f0998363af9" containerName="istio-proxy"
Sep 14 08:05:11.169794 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.169737    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="40d7fe21-740a-4a9a-b98a-3f0998363af9" containerName="istio-proxy"
Sep 14 08:05:11.169945 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.169831    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="40d7fe21-740a-4a9a-b98a-3f0998363af9" containerName="istio-proxy"
Sep 14 08:05:11.172912 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.172890    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.176151 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.176125    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-885430\"/\"openshift-service-ca.crt\""
Sep 14 08:05:11.176151 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.176146    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-885430\"/\"autores-model-885430-kserve-self-signed-certs\""
Sep 14 08:05:11.176375 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.176172    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-885430\"/\"kube-root-ca.crt\""
Sep 14 08:05:11.176375 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.176175    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-885430\"/\"default-dockercfg-wmjjk\""
Sep 14 08:05:11.179764 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.179743    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"]
Sep 14 08:05:11.280690 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.280651    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbhmz\" (UniqueName: \"kubernetes.io/projected/0a033d7d-41f3-4bac-acb5-caa54b460104-kube-api-access-gbhmz\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.280870 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.280729    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-model-cache\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.280870 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.280764    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-tmp-dir\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.280870 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.280839    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a033d7d-41f3-4bac-acb5-caa54b460104-tls-certs\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.280988 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.280875    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-dshm\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.280988 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.280912    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-home\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.382084 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.382037    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbhmz\" (UniqueName: \"kubernetes.io/projected/0a033d7d-41f3-4bac-acb5-caa54b460104-kube-api-access-gbhmz\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.382299 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.382134    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-model-cache\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.382299 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.382170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-tmp-dir\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.382299 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.382273    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a033d7d-41f3-4bac-acb5-caa54b460104-tls-certs\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.382444 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.382307    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-dshm\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.382444 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.382360    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-home\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.382607 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.382584    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-model-cache\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.382682 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.382645    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-tmp-dir\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.382740 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.382653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-home\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.384543 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.384517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-dshm\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.384898 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.384878    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a033d7d-41f3-4bac-acb5-caa54b460104-tls-certs\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.389642 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.389623    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbhmz\" (UniqueName: \"kubernetes.io/projected/0a033d7d-41f3-4bac-acb5-caa54b460104-kube-api-access-gbhmz\") pod \"autores-model-885430-kserve-5ddccd4bd7-hfsdr\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") " pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.483028 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.482937    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:11.673250 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.673192    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:11.758014 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:11.757922    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"]
Sep 14 08:05:12.550495 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:12.550459    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr" event={"ID":"0a033d7d-41f3-4bac-acb5-caa54b460104","Type":"ContainerStarted","Data":"0a2a0e5adf5acea3ebc7502b2235d5b10c9b783dc9edb444d0379d3b32d5ebe6"}
Sep 14 08:05:12.550846 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:12.550506    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr" event={"ID":"0a033d7d-41f3-4bac-acb5-caa54b460104","Type":"ContainerStarted","Data":"648c6d7a61e52fd8869b61b5cf1b3d07ecff194e39a73ebaf824600f2e0dd953"}
Sep 14 08:05:12.581753 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:12.581707    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr" podStartSLOduration=1.581693032 podStartE2EDuration="1.581693032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:05:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:05:12.581243037 +0000 UTC m=+967.285781540" watchObservedRunningTime="2026-09-14 08:05:12.581693032 +0000 UTC m=+967.286231534"
Sep 14 08:05:21.484116 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:21.484077    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:21.484551 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:21.484131    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:21.496970 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:21.496937    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:21.609957 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:21.609928    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:05:24.095104 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:24.095070    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:05:25.484664 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.484631    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"]
Sep 14 08:05:25.485162 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.485146    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="40d7fe21-740a-4a9a-b98a-3f0998363af9" containerName="istio-proxy"
Sep 14 08:05:25.487471 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.487454    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.489837 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.489815    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e7b0a8\"/\"kube-root-ca.crt\""
Sep 14 08:05:25.490803 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.490767    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e7b0a8\"/\"default-dockercfg-hlsbb\""
Sep 14 08:05:25.490803 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.490772    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e7b0a8\"/\"openshift-service-ca.crt\""
Sep 14 08:05:25.490994 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.490916    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e7b0a8\"/\"test-model-e7b0a8-kserve-self-signed-certs\""
Sep 14 08:05:25.497344 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.497324    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"]
Sep 14 08:05:25.514762 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.514719    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:25.614632 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.614607    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-dshm\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.614787 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.614646    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-home\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.614787 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.614666    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-tmp-dir\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.614787 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.614730    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4n8j\" (UniqueName: \"kubernetes.io/projected/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-kube-api-access-v4n8j\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.614787 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.614762    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-model-cache\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.615015 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.614809    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-tls-certs\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.715597 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.715566    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-tls-certs\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.715737 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.715630    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-dshm\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.715737 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.715662    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-home\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.715737 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.715681    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-tmp-dir\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.715737 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.715712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4n8j\" (UniqueName: \"kubernetes.io/projected/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-kube-api-access-v4n8j\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.715737 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.715739    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-model-cache\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.716077 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.716056    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-home\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.716137 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.716081    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-tmp-dir\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.716181 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.716136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-model-cache\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.717970 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.717950    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-dshm\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.718148 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.718131    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-tls-certs\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.723188 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.723165    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4n8j\" (UniqueName: \"kubernetes.io/projected/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-kube-api-access-v4n8j\") pod \"test-model-e7b0a8-kserve-6dd9c597dc-ggnjt\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") " pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:25.799400 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:25.799308    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:26.102767 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:26.102740    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"]
Sep 14 08:05:26.104543 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:05:26.104511    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ae8ac9a_9ec8_4f08_a0c1_1b5ce02ca460.slice/crio-a0a067186e35ba0337c145e0ba70f56c026b0cf45d5f4c561b716e3f7ecf3d98 WatchSource:0}: Error finding container a0a067186e35ba0337c145e0ba70f56c026b0cf45d5f4c561b716e3f7ecf3d98: Status 404 returned error can't find the container with id a0a067186e35ba0337c145e0ba70f56c026b0cf45d5f4c561b716e3f7ecf3d98
Sep 14 08:05:26.626963 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:26.626921    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt" event={"ID":"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460","Type":"ContainerStarted","Data":"bed01a35eea2bb07fbd3164d9e155ea0342e8942fd8ab2064d8b66bb4d5c174b"}
Sep 14 08:05:26.626963 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:26.626968    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt" event={"ID":"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460","Type":"ContainerStarted","Data":"a0a067186e35ba0337c145e0ba70f56c026b0cf45d5f4c561b716e3f7ecf3d98"}
Sep 14 08:05:26.641945 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:26.641899    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt" podStartSLOduration=1.641883413 podStartE2EDuration="1.641883413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:05:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:05:26.640734257 +0000 UTC m=+981.345272759" watchObservedRunningTime="2026-09-14 08:05:26.641883413 +0000 UTC m=+981.346421960"
Sep 14 08:05:27.483260 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:27.483224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:28.081311 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:28.081237    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:28.833578 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:28.833540    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:29.449981 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:29.449919    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:30.420769 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:30.420726    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:31.036594 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:31.036555    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:31.644110 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:31.644080    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:32.436370 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:32.436338    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:33.198166 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:33.198135    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:33.838644 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:33.838610    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:34.552879 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:34.552837    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:35.181800 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:35.181765    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:35.800378 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:35.800342    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:35.800378 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:35.800389    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:35.812917 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:35.812890    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:35.981837 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:35.981802    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:36.697634 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:36.697608    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:05:41.005331 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:41.005299    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:05:42.507352 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:42.507314    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:43.050078 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:43.049992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:43.135645 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:43.135607    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:43.250875 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:43.250841    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:43.314641 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:43.314536    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:43.375745 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:43.375717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:46.194279 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:46.194242    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:46.293142 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:46.293108    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:46.944694 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:46.944657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:47.560680 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:47.560651    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:48.348012 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:48.347982    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:49.959140 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:49.959099    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"]
Sep 14 08:05:49.976648 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:49.976616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:51.379310 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:51.379275    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:51.477897 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:51.477866    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:51.577385 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:51.577343    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:51.677842 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:51.677752    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:52.094264 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:52.094144    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:53.849933 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:53.849895    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:54.385983 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:54.385950    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:54.486400 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:54.486362    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:54.556456 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:54.556421    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:54.614883 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:54.614857    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:54.916923 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:54.916889    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:55.051803 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:55.051775    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:55.724234 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:55.724164    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:56.514620 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:56.514580    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:57.179761 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:57.179728    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:57.826353 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:57.826322    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:59.994146 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:05:59.994114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:01.389195 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:01.389146    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:06:03.100955 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:03.100921    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:04.978434 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:04.978383    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr" podUID="0a033d7d-41f3-4bac-acb5-caa54b460104" containerName="main" containerID="cri-o://0a2a0e5adf5acea3ebc7502b2235d5b10c9b783dc9edb444d0379d3b32d5ebe6" gracePeriod=45
Sep 14 08:06:05.224233 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.224181    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:06:05.301525 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.301445    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbhmz\" (UniqueName: \"kubernetes.io/projected/0a033d7d-41f3-4bac-acb5-caa54b460104-kube-api-access-gbhmz\") pod \"0a033d7d-41f3-4bac-acb5-caa54b460104\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") "
Sep 14 08:06:05.301525 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.301482    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-tmp-dir\") pod \"0a033d7d-41f3-4bac-acb5-caa54b460104\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") "
Sep 14 08:06:05.301525 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.301498    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-home\") pod \"0a033d7d-41f3-4bac-acb5-caa54b460104\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") "
Sep 14 08:06:05.301804 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.301550    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-model-cache\") pod \"0a033d7d-41f3-4bac-acb5-caa54b460104\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") "
Sep 14 08:06:05.301804 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.301577    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-dshm\") pod \"0a033d7d-41f3-4bac-acb5-caa54b460104\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") "
Sep 14 08:06:05.301804 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.301650    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a033d7d-41f3-4bac-acb5-caa54b460104-tls-certs\") pod \"0a033d7d-41f3-4bac-acb5-caa54b460104\" (UID: \"0a033d7d-41f3-4bac-acb5-caa54b460104\") "
Sep 14 08:06:05.301804 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.301725    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0a033d7d-41f3-4bac-acb5-caa54b460104" (UID: "0a033d7d-41f3-4bac-acb5-caa54b460104"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:06:05.301973 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.301824    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-home" (OuterVolumeSpecName: "home") pod "0a033d7d-41f3-4bac-acb5-caa54b460104" (UID: "0a033d7d-41f3-4bac-acb5-caa54b460104"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:06:05.301973 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.301840    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-model-cache" (OuterVolumeSpecName: "model-cache") pod "0a033d7d-41f3-4bac-acb5-caa54b460104" (UID: "0a033d7d-41f3-4bac-acb5-caa54b460104"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:06:05.302058 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.301998    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-tmp-dir\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:05.302058 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.302012    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-home\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:05.302058 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.302020    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-model-cache\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:05.303774 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.303742    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-dshm" (OuterVolumeSpecName: "dshm") pod "0a033d7d-41f3-4bac-acb5-caa54b460104" (UID: "0a033d7d-41f3-4bac-acb5-caa54b460104"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:06:05.303873 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.303844    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a033d7d-41f3-4bac-acb5-caa54b460104-kube-api-access-gbhmz" (OuterVolumeSpecName: "kube-api-access-gbhmz") pod "0a033d7d-41f3-4bac-acb5-caa54b460104" (UID: "0a033d7d-41f3-4bac-acb5-caa54b460104"). InnerVolumeSpecName "kube-api-access-gbhmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:06:05.303873 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.303863    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a033d7d-41f3-4bac-acb5-caa54b460104-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0a033d7d-41f3-4bac-acb5-caa54b460104" (UID: "0a033d7d-41f3-4bac-acb5-caa54b460104"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:06:05.402872 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.402846    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbhmz\" (UniqueName: \"kubernetes.io/projected/0a033d7d-41f3-4bac-acb5-caa54b460104-kube-api-access-gbhmz\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:05.402872 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.402871    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a033d7d-41f3-4bac-acb5-caa54b460104-dshm\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:05.402995 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.402882    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a033d7d-41f3-4bac-acb5-caa54b460104-tls-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:05.839335 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.839301    2576 generic.go:356] "Generic (PLEG): container finished" podID="0a033d7d-41f3-4bac-acb5-caa54b460104" containerID="0a2a0e5adf5acea3ebc7502b2235d5b10c9b783dc9edb444d0379d3b32d5ebe6" exitCode=0
Sep 14 08:06:05.839523 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.839405    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"
Sep 14 08:06:05.840926 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.840901    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr" event={"ID":"0a033d7d-41f3-4bac-acb5-caa54b460104","Type":"ContainerDied","Data":"0a2a0e5adf5acea3ebc7502b2235d5b10c9b783dc9edb444d0379d3b32d5ebe6"}
Sep 14 08:06:05.840926 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.840930    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr" event={"ID":"0a033d7d-41f3-4bac-acb5-caa54b460104","Type":"ContainerDied","Data":"648c6d7a61e52fd8869b61b5cf1b3d07ecff194e39a73ebaf824600f2e0dd953"}
Sep 14 08:06:05.841112 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.840948    2576 scope.go:117] "RemoveContainer" containerID="0a2a0e5adf5acea3ebc7502b2235d5b10c9b783dc9edb444d0379d3b32d5ebe6"
Sep 14 08:06:05.850843 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.850803    2576 scope.go:117] "RemoveContainer" containerID="0a2a0e5adf5acea3ebc7502b2235d5b10c9b783dc9edb444d0379d3b32d5ebe6"
Sep 14 08:06:05.851117 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:06:05.851096    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a2a0e5adf5acea3ebc7502b2235d5b10c9b783dc9edb444d0379d3b32d5ebe6\": container with ID starting with 0a2a0e5adf5acea3ebc7502b2235d5b10c9b783dc9edb444d0379d3b32d5ebe6 not found: ID does not exist" containerID="0a2a0e5adf5acea3ebc7502b2235d5b10c9b783dc9edb444d0379d3b32d5ebe6"
Sep 14 08:06:05.851184 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.851125    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a2a0e5adf5acea3ebc7502b2235d5b10c9b783dc9edb444d0379d3b32d5ebe6"} err="failed to get container status \"0a2a0e5adf5acea3ebc7502b2235d5b10c9b783dc9edb444d0379d3b32d5ebe6\": rpc error: code = NotFound desc = could not find container \"0a2a0e5adf5acea3ebc7502b2235d5b10c9b783dc9edb444d0379d3b32d5ebe6\": container with ID starting with 0a2a0e5adf5acea3ebc7502b2235d5b10c9b783dc9edb444d0379d3b32d5ebe6 not found: ID does not exist"
Sep 14 08:06:05.865952 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.865929    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"]
Sep 14 08:06:05.868643 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:05.868623    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-885430/autores-model-885430-kserve-5ddccd4bd7-hfsdr"]
Sep 14 08:06:07.842399 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:07.839990    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a033d7d-41f3-4bac-acb5-caa54b460104" path="/var/lib/kubelet/pods/0a033d7d-41f3-4bac-acb5-caa54b460104/volumes"
Sep 14 08:06:10.202321 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:10.202291    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:14.498124 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:14.498090    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:06:18.038038 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:18.037999    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:19.486544 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:19.486510    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:23.738763 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:23.738728    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:23.845560 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:23.845524    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:23.935859 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:23.935833    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:29.805171 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:29.805138    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:35.494413 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:35.494372    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:06:36.910982 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:36.910949    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:37.197776 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:37.197681    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"]
Sep 14 08:06:37.197992 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:37.197960    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy" containerID="cri-o://12dfaeb69b1a57fd173c5c9ba758b669c33105a1b71db29a859f9a4618b809eb" gracePeriod=30
Sep 14 08:06:37.209350 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:37.209328    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:37.632594 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:37.632563    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:37.676656 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:37.676624    2576 patch_prober.go:28] interesting pod/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 08:06:37.676776 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:37.676708    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:06:37.966723 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:37.966614    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:38.300866 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:38.300782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:39.831910 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:39.831877    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:40.929523 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:40.929483    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:41.165896 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:41.165856    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:41.266408 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:41.266326    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:41.863147 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:41.863114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:42.456241 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.456219    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:06:42.640651 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.640623    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-workload-socket\") pod \"271d49fc-4f91-466f-a1a3-800284dfa5d7\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") "
Sep 14 08:06:42.640786 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.640657    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-token\") pod \"271d49fc-4f91-466f-a1a3-800284dfa5d7\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") "
Sep 14 08:06:42.640786 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.640722    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-credential-socket\") pod \"271d49fc-4f91-466f-a1a3-800284dfa5d7\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") "
Sep 14 08:06:42.640786 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.640737    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-data\") pod \"271d49fc-4f91-466f-a1a3-800284dfa5d7\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") "
Sep 14 08:06:42.640786 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.640754    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54fvz\" (UniqueName: \"kubernetes.io/projected/271d49fc-4f91-466f-a1a3-800284dfa5d7-kube-api-access-54fvz\") pod \"271d49fc-4f91-466f-a1a3-800284dfa5d7\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") "
Sep 14 08:06:42.640786 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.640769    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-envoy\") pod \"271d49fc-4f91-466f-a1a3-800284dfa5d7\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") "
Sep 14 08:06:42.641038 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.640832    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-podinfo\") pod \"271d49fc-4f91-466f-a1a3-800284dfa5d7\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") "
Sep 14 08:06:42.641038 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.640878    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/271d49fc-4f91-466f-a1a3-800284dfa5d7-istiod-ca-cert\") pod \"271d49fc-4f91-466f-a1a3-800284dfa5d7\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") "
Sep 14 08:06:42.641038 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.640904    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-workload-certs\") pod \"271d49fc-4f91-466f-a1a3-800284dfa5d7\" (UID: \"271d49fc-4f91-466f-a1a3-800284dfa5d7\") "
Sep 14 08:06:42.641038 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.640905    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "271d49fc-4f91-466f-a1a3-800284dfa5d7" (UID: "271d49fc-4f91-466f-a1a3-800284dfa5d7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:06:42.641038 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.640995    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "271d49fc-4f91-466f-a1a3-800284dfa5d7" (UID: "271d49fc-4f91-466f-a1a3-800284dfa5d7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:06:42.641038 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.641028    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-data" (OuterVolumeSpecName: "istio-data") pod "271d49fc-4f91-466f-a1a3-800284dfa5d7" (UID: "271d49fc-4f91-466f-a1a3-800284dfa5d7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:06:42.641397 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.641292    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:42.641397 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.641305    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "271d49fc-4f91-466f-a1a3-800284dfa5d7" (UID: "271d49fc-4f91-466f-a1a3-800284dfa5d7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:06:42.641397 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.641313    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:42.641397 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.641347    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:42.641397 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.641345    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/271d49fc-4f91-466f-a1a3-800284dfa5d7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "271d49fc-4f91-466f-a1a3-800284dfa5d7" (UID: "271d49fc-4f91-466f-a1a3-800284dfa5d7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:06:42.642580 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.642537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:42.643128 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.643103    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-token" (OuterVolumeSpecName: "istio-token") pod "271d49fc-4f91-466f-a1a3-800284dfa5d7" (UID: "271d49fc-4f91-466f-a1a3-800284dfa5d7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:06:42.643333 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.643313    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/271d49fc-4f91-466f-a1a3-800284dfa5d7-kube-api-access-54fvz" (OuterVolumeSpecName: "kube-api-access-54fvz") pod "271d49fc-4f91-466f-a1a3-800284dfa5d7" (UID: "271d49fc-4f91-466f-a1a3-800284dfa5d7"). InnerVolumeSpecName "kube-api-access-54fvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:06:42.643475 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.643450    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "271d49fc-4f91-466f-a1a3-800284dfa5d7" (UID: "271d49fc-4f91-466f-a1a3-800284dfa5d7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:06:42.643539 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.643452    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "271d49fc-4f91-466f-a1a3-800284dfa5d7" (UID: "271d49fc-4f91-466f-a1a3-800284dfa5d7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:06:42.742845 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.742807    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54fvz\" (UniqueName: \"kubernetes.io/projected/271d49fc-4f91-466f-a1a3-800284dfa5d7-kube-api-access-54fvz\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:42.742845 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.742841    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:42.742845 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.742851    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:42.743078 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.742859    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/271d49fc-4f91-466f-a1a3-800284dfa5d7-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:42.743078 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.742868    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/271d49fc-4f91-466f-a1a3-800284dfa5d7-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:42.743078 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:42.742875    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/271d49fc-4f91-466f-a1a3-800284dfa5d7-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:06:43.040202 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:43.040112    2576 generic.go:356] "Generic (PLEG): container finished" podID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerID="12dfaeb69b1a57fd173c5c9ba758b669c33105a1b71db29a859f9a4618b809eb" exitCode=0
Sep 14 08:06:43.040202 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:43.040155    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" event={"ID":"271d49fc-4f91-466f-a1a3-800284dfa5d7","Type":"ContainerDied","Data":"12dfaeb69b1a57fd173c5c9ba758b669c33105a1b71db29a859f9a4618b809eb"}
Sep 14 08:06:43.040202 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:43.040163    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"
Sep 14 08:06:43.040418 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:43.040222    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm" event={"ID":"271d49fc-4f91-466f-a1a3-800284dfa5d7","Type":"ContainerDied","Data":"3b5aa9cc01e2c3734990eb931ca0f5704ea1050a1aed17a83696c28d2db2ee51"}
Sep 14 08:06:43.040418 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:43.040248    2576 scope.go:117] "RemoveContainer" containerID="12dfaeb69b1a57fd173c5c9ba758b669c33105a1b71db29a859f9a4618b809eb"
Sep 14 08:06:43.050192 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:43.050174    2576 scope.go:117] "RemoveContainer" containerID="0e4f52e31112dc04b0253886a81fe9f99668226aa49724718574635562ec666c"
Sep 14 08:06:43.058258 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:43.058241    2576 scope.go:117] "RemoveContainer" containerID="12dfaeb69b1a57fd173c5c9ba758b669c33105a1b71db29a859f9a4618b809eb"
Sep 14 08:06:43.058497 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:06:43.058481    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12dfaeb69b1a57fd173c5c9ba758b669c33105a1b71db29a859f9a4618b809eb\": container with ID starting with 12dfaeb69b1a57fd173c5c9ba758b669c33105a1b71db29a859f9a4618b809eb not found: ID does not exist" containerID="12dfaeb69b1a57fd173c5c9ba758b669c33105a1b71db29a859f9a4618b809eb"
Sep 14 08:06:43.058549 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:43.058504    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12dfaeb69b1a57fd173c5c9ba758b669c33105a1b71db29a859f9a4618b809eb"} err="failed to get container status \"12dfaeb69b1a57fd173c5c9ba758b669c33105a1b71db29a859f9a4618b809eb\": rpc error: code = NotFound desc = could not find container \"12dfaeb69b1a57fd173c5c9ba758b669c33105a1b71db29a859f9a4618b809eb\": container with ID starting with 12dfaeb69b1a57fd173c5c9ba758b669c33105a1b71db29a859f9a4618b809eb not found: ID does not exist"
Sep 14 08:06:43.058549 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:43.058520    2576 scope.go:117] "RemoveContainer" containerID="0e4f52e31112dc04b0253886a81fe9f99668226aa49724718574635562ec666c"
Sep 14 08:06:43.058773 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:06:43.058749    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e4f52e31112dc04b0253886a81fe9f99668226aa49724718574635562ec666c\": container with ID starting with 0e4f52e31112dc04b0253886a81fe9f99668226aa49724718574635562ec666c not found: ID does not exist" containerID="0e4f52e31112dc04b0253886a81fe9f99668226aa49724718574635562ec666c"
Sep 14 08:06:43.058845 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:43.058781    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e4f52e31112dc04b0253886a81fe9f99668226aa49724718574635562ec666c"} err="failed to get container status \"0e4f52e31112dc04b0253886a81fe9f99668226aa49724718574635562ec666c\": rpc error: code = NotFound desc = could not find container \"0e4f52e31112dc04b0253886a81fe9f99668226aa49724718574635562ec666c\": container with ID starting with 0e4f52e31112dc04b0253886a81fe9f99668226aa49724718574635562ec666c not found: ID does not exist"
Sep 14 08:06:43.061714 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:43.061691    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"]
Sep 14 08:06:43.065019 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:43.064996    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-885430-openshift-default-7b47c7f9d6-wxkxm"]
Sep 14 08:06:43.440422 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:43.440388    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:43.839246 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:43.839156    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" path="/var/lib/kubelet/pods/271d49fc-4f91-466f-a1a3-800284dfa5d7/volumes"
Sep 14 08:06:44.902351 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:44.902317    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:45.002993 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:45.002958    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:45.589307 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:45.589274    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:45.691505 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:45.691475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:45.790034 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:45.790003    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:56.362376 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:56.362339    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:57.226053 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:06:57.226023    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:03.131241 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:03.131187    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:03.194364 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:03.194331    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:03.274803 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:03.274767    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:03.381367 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:03.381306    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:03.499481 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:03.499455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:04.114598 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:04.114567    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:04.212031 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:04.211998    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:04.316028 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:04.316001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:05.870902 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:05.870874    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:07.028819 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:07.028783    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:07.676508 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:07.676475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:08.537888 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:08.537854    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:08.699242 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:08.699197    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:07:09.274900 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:09.274859    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:09.910126 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:09.910094    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:10.730969 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:10.730928    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:11.347631 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:11.347595    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:11.634783 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:11.634749    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:11.735249 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:11.735194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:12.325189 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:12.325158    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:18.017951 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.017922    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:18.079668 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.079639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:18.235733 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.235705    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:18.514971 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.514939    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"]
Sep 14 08:07:18.515515 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.515489    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a033d7d-41f3-4bac-acb5-caa54b460104" containerName="main"
Sep 14 08:07:18.515660 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.515555    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a033d7d-41f3-4bac-acb5-caa54b460104" containerName="main"
Sep 14 08:07:18.515660 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.515625    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy"
Sep 14 08:07:18.515660 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.515635    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy"
Sep 14 08:07:18.515660 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.515650    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy"
Sep 14 08:07:18.515942 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.515666    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy"
Sep 14 08:07:18.515942 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.515792    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a033d7d-41f3-4bac-acb5-caa54b460104" containerName="main"
Sep 14 08:07:18.515942 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.515811    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy"
Sep 14 08:07:18.515942 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.515823    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="271d49fc-4f91-466f-a1a3-800284dfa5d7" containerName="istio-proxy"
Sep 14 08:07:18.519254 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.519232    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.523451 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.523425    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-58a5b128-openshift-default-dockercfg-kpl78\""
Sep 14 08:07:18.531642 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.531619    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"]
Sep 14 08:07:18.547464 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.547433    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:18.678025 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.677989    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5spvb\" (UniqueName: \"kubernetes.io/projected/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-kube-api-access-5spvb\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.678190 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.678034    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-workload-socket\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.678190 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.678068    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-token\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.678190 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.678123    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-workload-certs\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.678190 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.678151    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-envoy\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.678190 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.678169    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istiod-ca-cert\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.678372 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.678273    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-podinfo\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.678372 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.678308    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-credential-socket\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.678372 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.678335    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-data\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.779879 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.779802    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-envoy\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.779879 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.779835    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istiod-ca-cert\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.779879 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.779868    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-podinfo\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.780122 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.779896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-credential-socket\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.780122 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.779924    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-data\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.780122 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.779986    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5spvb\" (UniqueName: \"kubernetes.io/projected/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-kube-api-access-5spvb\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.780122 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.780015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-workload-socket\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.780122 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.780054    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-token\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.780122 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.780092    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-workload-certs\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.780481 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.780377    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-credential-socket\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.780481 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.780456    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-data\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.780568 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.780477    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-workload-socket\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.780568 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.780492    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-workload-certs\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.780694 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.780676    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istiod-ca-cert\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.782140 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.782121    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-envoy\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.782602 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.782584    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-podinfo\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.788119 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.788090    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5spvb\" (UniqueName: \"kubernetes.io/projected/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-kube-api-access-5spvb\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.788594 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.788574    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-token\") pod \"e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") " pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.831733 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.831710    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:18.895474 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:18.895440    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:19.183243 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:07:19.183187    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0a9b2fa_f6eb_47c7_af69_b84ff8cc2053.slice/crio-5b8efb746a88e34190a887b71247e058aeacb284b053883d168bcfc369e3c4bb WatchSource:0}: Error finding container 5b8efb746a88e34190a887b71247e058aeacb284b053883d168bcfc369e3c4bb: Status 404 returned error can't find the container with id 5b8efb746a88e34190a887b71247e058aeacb284b053883d168bcfc369e3c4bb
Sep 14 08:07:19.186227 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:19.186166    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:07:19.186335 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:19.186283    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:07:19.186384 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:19.186335    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:07:19.186530 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:19.186506    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"]
Sep 14 08:07:19.222744 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:19.222721    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp" event={"ID":"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053","Type":"ContainerStarted","Data":"5b8efb746a88e34190a887b71247e058aeacb284b053883d168bcfc369e3c4bb"}
Sep 14 08:07:20.235777 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:20.235742    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp" event={"ID":"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053","Type":"ContainerStarted","Data":"ebc4a7201350b865dc3aa3ad00b2ad18bae7a900dfe10a68781c6a6462b399c5"}
Sep 14 08:07:20.271033 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:20.270977    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp" podStartSLOduration=2.270961852 podStartE2EDuration="2.270961852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:20.268380905 +0000 UTC m=+1094.972919406" watchObservedRunningTime="2026-09-14 08:07:20.270961852 +0000 UTC m=+1094.975500357"
Sep 14 08:07:20.832877 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:20.832841    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:20.838317 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:20.838290    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:21.241459 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:21.241431    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:21.242181 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:21.242163    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:23.339605 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:23.339575    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:23.990481 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:23.990450    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:24.230088 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:24.230054    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:24.322532 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:24.322448    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:25.171750 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:25.171720    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:26.060763 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:26.060727    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:26.731593 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:26.731563    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:27.803746 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:27.803714    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:28.143228 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:28.143174    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:28.243325 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:28.243288    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:28.343228 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:28.343180    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:28.584186 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:28.584094    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:28.678915 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:28.678883    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:32.701889 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:32.701856    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:07:32.886479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:32.886448    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:32.985347 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:32.985265    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx"]
Sep 14 08:07:32.988449 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:32.988433    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx"
Sep 14 08:07:32.990742 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:32.990720    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:07:32.995701 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:32.995676    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx"]
Sep 14 08:07:33.019138 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:33.019113    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e8f853-d4f6-4e44-838c-1c243b243c85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-239599-6chzx\" (UID: \"32e8f853-d4f6-4e44-838c-1c243b243c85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx"
Sep 14 08:07:33.120812 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:33.120770    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e8f853-d4f6-4e44-838c-1c243b243c85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-239599-6chzx\" (UID: \"32e8f853-d4f6-4e44-838c-1c243b243c85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx"
Sep 14 08:07:33.121464 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:33.121439    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e8f853-d4f6-4e44-838c-1c243b243c85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-239599-6chzx\" (UID: \"32e8f853-d4f6-4e44-838c-1c243b243c85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx"
Sep 14 08:07:33.302345 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:33.302271    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx"
Sep 14 08:07:33.320272 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:33.320238    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:33.573003 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:33.572976    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx"]
Sep 14 08:07:33.575478 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:07:33.575454    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e8f853_d4f6_4e44_838c_1c243b243c85.slice/crio-73ad45f54e7fe7eca73834033b313f739e9b4b101f8840122e9473311cb1b5d1 WatchSource:0}: Error finding container 73ad45f54e7fe7eca73834033b313f739e9b4b101f8840122e9473311cb1b5d1: Status 404 returned error can't find the container with id 73ad45f54e7fe7eca73834033b313f739e9b4b101f8840122e9473311cb1b5d1
Sep 14 08:07:33.578032 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:33.578015    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:07:34.086712 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:34.086681    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:34.309720 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:34.309686    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx" event={"ID":"32e8f853-d4f6-4e44-838c-1c243b243c85","Type":"ContainerStarted","Data":"73ad45f54e7fe7eca73834033b313f739e9b4b101f8840122e9473311cb1b5d1"}
Sep 14 08:07:34.951759 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:34.951723    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:35.636249 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:35.633759    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:36.249163 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:36.249094    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:36.920781 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:36.920640    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:37.331512 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:37.331425    2576 generic.go:356] "Generic (PLEG): container finished" podID="32e8f853-d4f6-4e44-838c-1c243b243c85" containerID="b385f3f9ccec3a0aa24eb6af5738bb488c45e73b765c2532943e59c76a2fc78d" exitCode=0
Sep 14 08:07:37.331741 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:37.331718    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx" event={"ID":"32e8f853-d4f6-4e44-838c-1c243b243c85","Type":"ContainerDied","Data":"b385f3f9ccec3a0aa24eb6af5738bb488c45e73b765c2532943e59c76a2fc78d"}
Sep 14 08:07:37.648983 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:37.648945    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:38.402599 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:38.402546    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:38.509625 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:38.509603    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx"
Sep 14 08:07:38.580909 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:38.580881    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e8f853-d4f6-4e44-838c-1c243b243c85-maas-api-service-ca\") pod \"32e8f853-d4f6-4e44-838c-1c243b243c85\" (UID: \"32e8f853-d4f6-4e44-838c-1c243b243c85\") "
Sep 14 08:07:38.581250 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:38.581204    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32e8f853-d4f6-4e44-838c-1c243b243c85-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "32e8f853-d4f6-4e44-838c-1c243b243c85" (UID: "32e8f853-d4f6-4e44-838c-1c243b243c85"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:38.682504 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:38.682475    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e8f853-d4f6-4e44-838c-1c243b243c85-maas-api-service-ca\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:07:38.768687 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:38.768654    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:39.135967 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:39.135906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:39.348639 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:39.348604    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx" event={"ID":"32e8f853-d4f6-4e44-838c-1c243b243c85","Type":"ContainerDied","Data":"73ad45f54e7fe7eca73834033b313f739e9b4b101f8840122e9473311cb1b5d1"}
Sep 14 08:07:39.348897 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:39.348672    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73ad45f54e7fe7eca73834033b313f739e9b4b101f8840122e9473311cb1b5d1"
Sep 14 08:07:39.348897 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:39.348615    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx"
Sep 14 08:07:39.443705 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:39.443605    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:39.540293 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:39.540114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:39.865724 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:39.865693    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:39.967979 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:39.967950    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:40.620973 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:40.620945    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:41.430048 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:41.430011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:42.077930 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.077883    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"]
Sep 14 08:07:42.078549 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.078513    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32e8f853-d4f6-4e44-838c-1c243b243c85" containerName="revoke-keys"
Sep 14 08:07:42.078549 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.078537    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="32e8f853-d4f6-4e44-838c-1c243b243c85" containerName="revoke-keys"
Sep 14 08:07:42.078724 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.078667    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="32e8f853-d4f6-4e44-838c-1c243b243c85" containerName="revoke-keys"
Sep 14 08:07:42.081906 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.081888    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.084115 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.084096    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"openshift-service-ca.crt\""
Sep 14 08:07:42.084323 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.084301    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"kube-root-ca.crt\""
Sep 14 08:07:42.085232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.085192    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"default-dockercfg-4p5xb\""
Sep 14 08:07:42.085341 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.085317    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"e2e-conflict-a-2827d38c-kserve-self-signed-certs\""
Sep 14 08:07:42.093975 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.093953    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"]
Sep 14 08:07:42.105022 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.105001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:42.118919 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.118895    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-home\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.119015 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.118968    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-model-cache\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.119073 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.119044    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5gsk\" (UniqueName: \"kubernetes.io/projected/a7610e87-7025-4ae2-b822-df0d0bff88cb-kube-api-access-z5gsk\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.119123 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.119108    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-tmp-dir\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.119177 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.119139    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-dshm\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.119258 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.119178    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7610e87-7025-4ae2-b822-df0d0bff88cb-tls-certs\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.200893 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.200866    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:42.220036 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.220009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-home\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.220128 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.220065    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-model-cache\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.220128 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.220110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5gsk\" (UniqueName: \"kubernetes.io/projected/a7610e87-7025-4ae2-b822-df0d0bff88cb-kube-api-access-z5gsk\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.220250 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.220166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-tmp-dir\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.220314 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.220249    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-dshm\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.220314 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.220289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7610e87-7025-4ae2-b822-df0d0bff88cb-tls-certs\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.220502 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.220483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-home\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.220732 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.220698    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-model-cache\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.220852 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.220838    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-tmp-dir\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.222563 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.222543    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-dshm\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.223068 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.223049    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7610e87-7025-4ae2-b822-df0d0bff88cb-tls-certs\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.229791 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.229767    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5gsk\" (UniqueName: \"kubernetes.io/projected/a7610e87-7025-4ae2-b822-df0d0bff88cb-kube-api-access-z5gsk\") pod \"e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.394556 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.394525    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:42.667878 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.667854    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"]
Sep 14 08:07:42.670790 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:07:42.670762    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7610e87_7025_4ae2_b822_df0d0bff88cb.slice/crio-d4758c881ffbdd067d8f52e98aff2c73f497b766ad7946977b4bdf8a59dfb502 WatchSource:0}: Error finding container d4758c881ffbdd067d8f52e98aff2c73f497b766ad7946977b4bdf8a59dfb502: Status 404 returned error can't find the container with id d4758c881ffbdd067d8f52e98aff2c73f497b766ad7946977b4bdf8a59dfb502
Sep 14 08:07:42.843991 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:42.843956    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:43.371955 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:43.371927    2576 generic.go:356] "Generic (PLEG): container finished" podID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerID="ebc4a7201350b865dc3aa3ad00b2ad18bae7a900dfe10a68781c6a6462b399c5" exitCode=0
Sep 14 08:07:43.372371 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:43.371998    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp" event={"ID":"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053","Type":"ContainerDied","Data":"ebc4a7201350b865dc3aa3ad00b2ad18bae7a900dfe10a68781c6a6462b399c5"}
Sep 14 08:07:43.372534 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:43.372515    2576 scope.go:117] "RemoveContainer" containerID="ebc4a7201350b865dc3aa3ad00b2ad18bae7a900dfe10a68781c6a6462b399c5"
Sep 14 08:07:43.373730 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:43.373695    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:07:43.373831 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:43.373754    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg" event={"ID":"a7610e87-7025-4ae2-b822-df0d0bff88cb","Type":"ContainerStarted","Data":"a30c1456db5a2cfbe2b0a818a557ff65b57fa2222a1fa2230a87a7156359598a"}
Sep 14 08:07:43.373831 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:43.373773    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:07:43.373831 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:43.373812    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:07:43.373957 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:43.373780    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg" event={"ID":"a7610e87-7025-4ae2-b822-df0d0bff88cb","Type":"ContainerStarted","Data":"d4758c881ffbdd067d8f52e98aff2c73f497b766ad7946977b4bdf8a59dfb502"}
Sep 14 08:07:43.411168 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:43.411117    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg" podStartSLOduration=1.411097425 podStartE2EDuration="1.411097425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:43.406899476 +0000 UTC m=+1118.111437979" watchObservedRunningTime="2026-09-14 08:07:43.411097425 +0000 UTC m=+1118.115635929"
Sep 14 08:07:43.694260 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:43.694136    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx"]
Sep 14 08:07:43.697450 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:43.697426    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-239599-6chzx"]
Sep 14 08:07:43.839762 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:43.839730    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32e8f853-d4f6-4e44-838c-1c243b243c85" path="/var/lib/kubelet/pods/32e8f853-d4f6-4e44-838c-1c243b243c85/volumes"
Sep 14 08:07:43.962460 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:43.962366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:44.062504 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:44.062468    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:44.163536 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:44.163502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:44.381275 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:44.381241    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp" event={"ID":"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053","Type":"ContainerStarted","Data":"1de303261ba3ffa83c67aa13ffb0530d9b17c4bcd122f7f962c03bb6a6507f10"}
Sep 14 08:07:44.833738 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:44.833650    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:44.834332 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:44.834306    2576 patch_prober.go:28] interesting pod/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 14 08:07:44.834532 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:44.834506    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp" podUID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 14 08:07:44.978538 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:44.978510    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:45.736426 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:45.736381    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:45.832326 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:45.832289    2576 patch_prober.go:28] interesting pod/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 14 08:07:45.832520 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:45.832360    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp" podUID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 14 08:07:46.356101 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:46.356048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:46.417881 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:46.417848    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:46.543022 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:46.542991    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:46.842703 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:46.842620    2576 patch_prober.go:28] interesting pod/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 14 08:07:46.843268 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:46.843234    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp" podUID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 14 08:07:47.403472 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:47.403437    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:47.836667 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:47.836577    2576 patch_prober.go:28] interesting pod/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 14 08:07:47.836667 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:47.836654    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp" podUID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 14 08:07:48.131184 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:48.130197    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:48.832489 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:48.832448    2576 patch_prober.go:28] interesting pod/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 14 08:07:48.832696 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:48.832512    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp" podUID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 14 08:07:48.838187 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:48.837704    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:49.476060 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:49.476010    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:49.839729 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:49.839645    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:49.840088 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:49.840059    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:49.840680 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:49.840660    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:07:50.252893 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:50.252858    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:50.352990 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:50.352961    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:50.578992 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:50.578888    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:51.055576 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:51.055542    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:51.195139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:51.195104    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:51.525270 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:51.525235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:51.736737 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:51.736711    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:51.979463 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:51.979422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:52.395625 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:52.395599    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:52.395793 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:52.395640    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:52.408336 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:52.408316    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:52.434034 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:52.434013    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:07:53.026527 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:53.026492    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:53.135504 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:53.135470    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:53.284788 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:53.284706    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:56.639115 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:56.639083    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:56.742033 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:56.742001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:56.959562 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:56.959494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:57.061401 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:57.061372    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:57.344673 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:57.344597    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:58.016699 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:58.016663    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:58.752423 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:58.752383    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:59.467326 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:07:59.467292    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:02.304063 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:02.304021    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:02.405122 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:02.405085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:02.553589 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:02.553554    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"]
Sep 14 08:08:02.553849 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:02.553813    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp" podUID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerName="istio-proxy" containerID="cri-o://1de303261ba3ffa83c67aa13ffb0530d9b17c4bcd122f7f962c03bb6a6507f10" gracePeriod=30
Sep 14 08:08:02.573339 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:02.573315    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:02.934835 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:02.934800    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:03.029013 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:03.028983    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:03.148578 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:03.148542    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:04.841621 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:04.841585    2576 patch_prober.go:28] interesting pod/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 14 08:08:04.841972 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:04.841641    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp" podUID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:08:07.827312 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.827248    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:08:07.980841 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.980810    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-workload-socket\") pod \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") "
Sep 14 08:08:07.980988 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.980854    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-envoy\") pod \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") "
Sep 14 08:08:07.980988 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.980873    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istiod-ca-cert\") pod \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") "
Sep 14 08:08:07.980988 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.980955    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-workload-certs\") pod \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") "
Sep 14 08:08:07.980988 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.980981    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-credential-socket\") pod \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") "
Sep 14 08:08:07.981222 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.981013    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5spvb\" (UniqueName: \"kubernetes.io/projected/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-kube-api-access-5spvb\") pod \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") "
Sep 14 08:08:07.981222 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.981087    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-podinfo\") pod \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") "
Sep 14 08:08:07.981222 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.981122    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" (UID: "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:07.981222 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.981144    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-token\") pod \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") "
Sep 14 08:08:07.981222 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.981193    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-data\") pod \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\" (UID: \"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053\") "
Sep 14 08:08:07.981513 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.981322    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" (UID: "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:07.981513 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.981348    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" (UID: "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:08:07.981621 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.981551    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-data" (OuterVolumeSpecName: "istio-data") pod "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" (UID: "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:07.981695 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.981673    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" (UID: "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:07.981784 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.981766    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:08:07.981834 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.981791    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:08:07.981834 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.981805    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:08:07.981834 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.981817    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:08:07.983395 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.983364    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-kube-api-access-5spvb" (OuterVolumeSpecName: "kube-api-access-5spvb") pod "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" (UID: "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053"). InnerVolumeSpecName "kube-api-access-5spvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:08:07.983668 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.983647    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" (UID: "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:07.983750 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.983660    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" (UID: "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:08:07.983996 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:07.983975    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-token" (OuterVolumeSpecName: "istio-token") pod "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" (UID: "b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:08:08.082891 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.082808    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:08:08.082891 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.082839    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:08:08.082891 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.082848    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:08:08.082891 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.082857    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5spvb\" (UniqueName: \"kubernetes.io/projected/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-kube-api-access-5spvb\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:08:08.082891 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.082866    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:08:08.505737 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.505698    2576 generic.go:356] "Generic (PLEG): container finished" podID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerID="1de303261ba3ffa83c67aa13ffb0530d9b17c4bcd122f7f962c03bb6a6507f10" exitCode=0
Sep 14 08:08:08.505883 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.505770    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"
Sep 14 08:08:08.505883 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.505781    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp" event={"ID":"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053","Type":"ContainerDied","Data":"1de303261ba3ffa83c67aa13ffb0530d9b17c4bcd122f7f962c03bb6a6507f10"}
Sep 14 08:08:08.505883 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.505862    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp" event={"ID":"b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053","Type":"ContainerDied","Data":"5b8efb746a88e34190a887b71247e058aeacb284b053883d168bcfc369e3c4bb"}
Sep 14 08:08:08.506013 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.505887    2576 scope.go:117] "RemoveContainer" containerID="1de303261ba3ffa83c67aa13ffb0530d9b17c4bcd122f7f962c03bb6a6507f10"
Sep 14 08:08:08.515181 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.515164    2576 scope.go:117] "RemoveContainer" containerID="ebc4a7201350b865dc3aa3ad00b2ad18bae7a900dfe10a68781c6a6462b399c5"
Sep 14 08:08:08.524933 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.524907    2576 scope.go:117] "RemoveContainer" containerID="1de303261ba3ffa83c67aa13ffb0530d9b17c4bcd122f7f962c03bb6a6507f10"
Sep 14 08:08:08.525263 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:08:08.525240    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1de303261ba3ffa83c67aa13ffb0530d9b17c4bcd122f7f962c03bb6a6507f10\": container with ID starting with 1de303261ba3ffa83c67aa13ffb0530d9b17c4bcd122f7f962c03bb6a6507f10 not found: ID does not exist" containerID="1de303261ba3ffa83c67aa13ffb0530d9b17c4bcd122f7f962c03bb6a6507f10"
Sep 14 08:08:08.525354 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.525273    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1de303261ba3ffa83c67aa13ffb0530d9b17c4bcd122f7f962c03bb6a6507f10"} err="failed to get container status \"1de303261ba3ffa83c67aa13ffb0530d9b17c4bcd122f7f962c03bb6a6507f10\": rpc error: code = NotFound desc = could not find container \"1de303261ba3ffa83c67aa13ffb0530d9b17c4bcd122f7f962c03bb6a6507f10\": container with ID starting with 1de303261ba3ffa83c67aa13ffb0530d9b17c4bcd122f7f962c03bb6a6507f10 not found: ID does not exist"
Sep 14 08:08:08.525354 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.525298    2576 scope.go:117] "RemoveContainer" containerID="ebc4a7201350b865dc3aa3ad00b2ad18bae7a900dfe10a68781c6a6462b399c5"
Sep 14 08:08:08.525560 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:08:08.525545    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebc4a7201350b865dc3aa3ad00b2ad18bae7a900dfe10a68781c6a6462b399c5\": container with ID starting with ebc4a7201350b865dc3aa3ad00b2ad18bae7a900dfe10a68781c6a6462b399c5 not found: ID does not exist" containerID="ebc4a7201350b865dc3aa3ad00b2ad18bae7a900dfe10a68781c6a6462b399c5"
Sep 14 08:08:08.525605 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.525564    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebc4a7201350b865dc3aa3ad00b2ad18bae7a900dfe10a68781c6a6462b399c5"} err="failed to get container status \"ebc4a7201350b865dc3aa3ad00b2ad18bae7a900dfe10a68781c6a6462b399c5\": rpc error: code = NotFound desc = could not find container \"ebc4a7201350b865dc3aa3ad00b2ad18bae7a900dfe10a68781c6a6462b399c5\": container with ID starting with ebc4a7201350b865dc3aa3ad00b2ad18bae7a900dfe10a68781c6a6462b399c5 not found: ID does not exist"
Sep 14 08:08:08.527428 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.527409    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"]
Sep 14 08:08:08.531007 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.530982    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-58a5b128-openshift-default-656dfd9cdd-hc6bp"]
Sep 14 08:08:08.944462 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.944426    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:08.995528 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:08.995497    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:08:09.055086 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:09.055053    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:09.158048 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:09.158007    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:09.337281 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:09.337165    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:09.838693 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:09.838662    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" path="/var/lib/kubelet/pods/b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053/volumes"
Sep 14 08:08:15.178428 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:15.178395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:15.244055 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:15.244023    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:15.455857 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:15.455777    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:16.109769 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.109736    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:16.415399 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.415316    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:16.944026 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.943993    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"]
Sep 14 08:08:16.944469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.944454    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerName="istio-proxy"
Sep 14 08:08:16.944521 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.944471    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerName="istio-proxy"
Sep 14 08:08:16.944521 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.944482    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerName="istio-proxy"
Sep 14 08:08:16.944521 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.944488    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerName="istio-proxy"
Sep 14 08:08:16.944663 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.944560    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerName="istio-proxy"
Sep 14 08:08:16.950356 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.950327    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:16.954383 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.954344    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-1af772\"/\"openshift-service-ca.crt\""
Sep 14 08:08:16.954958 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.954714    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-1af772\"/\"default-dockercfg-lszmd\""
Sep 14 08:08:16.955262 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.955242    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-1af772\"/\"kube-root-ca.crt\""
Sep 14 08:08:16.958629 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.958609    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-1af772\"/\"noten-model-1af772-kserve-self-signed-certs\""
Sep 14 08:08:16.964547 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.964526    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-home\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:16.964644 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.964571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-model-cache\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:16.964644 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.964598    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-dshm\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:16.964737 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.964651    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-tls-certs\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:16.964737 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.964668    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-tmp-dir\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:16.964803 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.964751    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp7lb\" (UniqueName: \"kubernetes.io/projected/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-kube-api-access-tp7lb\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:16.965729 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.965707    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"]
Sep 14 08:08:16.967966 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:16.967944    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:17.065766 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.065726    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-home\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:17.065918 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.065781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-model-cache\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:17.065918 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.065890    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-dshm\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:17.066002 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.065935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-tls-certs\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:17.066002 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.065954    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-tmp-dir\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:17.066002 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.065992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp7lb\" (UniqueName: \"kubernetes.io/projected/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-kube-api-access-tp7lb\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:17.066173 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.066153    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-model-cache\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:17.066271 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.066234    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-home\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:17.066423 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.066403    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-tmp-dir\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:17.068157 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.068135    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-dshm\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:17.068407 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.068389    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-tls-certs\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:17.068637 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.068617    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:17.073727 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.073705    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp7lb\" (UniqueName: \"kubernetes.io/projected/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-kube-api-access-tp7lb\") pod \"noten-model-1af772-kserve-6f5dc96d95-vznx6\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") " pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:17.262333 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.262249    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:17.540088 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.540027    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"]
Sep 14 08:08:17.542749 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:08:17.542721    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b6babd7_f292_4b5c_8fbc_78401b81e5ad.slice/crio-dfa376067b7f2cd7927fd3a54ed283894fc97814feaa989dc411d5ea3a17bf20 WatchSource:0}: Error finding container dfa376067b7f2cd7927fd3a54ed283894fc97814feaa989dc411d5ea3a17bf20: Status 404 returned error can't find the container with id dfa376067b7f2cd7927fd3a54ed283894fc97814feaa989dc411d5ea3a17bf20
Sep 14 08:08:17.558052 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.558026    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6" event={"ID":"8b6babd7-f292-4b5c-8fbc-78401b81e5ad","Type":"ContainerStarted","Data":"dfa376067b7f2cd7927fd3a54ed283894fc97814feaa989dc411d5ea3a17bf20"}
Sep 14 08:08:17.944027 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:17.943990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:18.565286 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:18.565250    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6" event={"ID":"8b6babd7-f292-4b5c-8fbc-78401b81e5ad","Type":"ContainerStarted","Data":"88f7aaceafc66f57b9ea93d56168371d253576202333f2cedb39a3c863590032"}
Sep 14 08:08:18.581096 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:18.581043    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6" podStartSLOduration=2.581024721 podStartE2EDuration="2.581024721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:18.58048007 +0000 UTC m=+1153.285018572" watchObservedRunningTime="2026-09-14 08:08:18.581024721 +0000 UTC m=+1153.285563225"
Sep 14 08:08:19.024077 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:19.024043    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:19.669059 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:19.669022    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:19.772203 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:19.772160    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:19.870741 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:19.870704    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:20.541594 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:20.541535    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:21.278942 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:21.278913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:22.637455 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:22.637423    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:22.739004 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:22.738954    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:22.839161 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:22.839130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:23.102767 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:23.102674    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:23.431522 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:23.431482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:23.530959 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:23.530921    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:27.263392 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:27.263354    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:27.263392 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:27.263402    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:27.276049 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:27.276027    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:27.621566 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:27.621534    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:08:39.607376 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:39.607340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:39.700975 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:39.700932    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:08:43.648316 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:43.648278    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:44.380537 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:44.380497    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:45.603944 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:45.603913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:46.330611 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:46.330578    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:47.061514 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:47.061482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:47.511854 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:47.511826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:47.924378 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:47.924343    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:48.635486 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:48.635452    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:49.436585 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:49.436552    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:49.655875 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:49.655842    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"]
Sep 14 08:08:49.674764 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:49.674736    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:49.796232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:49.796120    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:50.197550 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.197519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:50.414916 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.414877    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"]
Sep 14 08:08:50.415637 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.415615    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0a9b2fa-f6eb-47c7-af69-b84ff8cc2053" containerName="istio-proxy"
Sep 14 08:08:50.419194 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.419175    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.421518 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.421496    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-4d8f28-openshift-default-dockercfg-htthj\""
Sep 14 08:08:50.429883 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.429861    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"]
Sep 14 08:08:50.461018 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.460937    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:50.493265 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.493234    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-envoy\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.493416 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.493270    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c897f\" (UniqueName: \"kubernetes.io/projected/c303423d-4fd4-4871-a852-acae8fb0e1f8-kube-api-access-c897f\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.493416 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.493307    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-workload-certs\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.493416 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.493374    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c303423d-4fd4-4871-a852-acae8fb0e1f8-istiod-ca-cert\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.493525 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.493435    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-credential-socket\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.493525 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.493454    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-workload-socket\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.493525 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.493489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-token\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.493525 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.493520    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-data\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.493651 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.493587    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-podinfo\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.594146 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.594108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-workload-certs\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.594146 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.594154    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c303423d-4fd4-4871-a852-acae8fb0e1f8-istiod-ca-cert\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.594456 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.594232    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-credential-socket\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.594456 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.594251    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-workload-socket\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.594456 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.594281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-token\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.594633 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.594459    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-data\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.594633 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.594591    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-podinfo\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.594633 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.594615    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-workload-certs\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.594633 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.594631    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-envoy\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.594812 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.594660    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c897f\" (UniqueName: \"kubernetes.io/projected/c303423d-4fd4-4871-a852-acae8fb0e1f8-kube-api-access-c897f\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.594812 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.594756    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-workload-socket\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.594881 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.594862    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-data\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.594952 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.594936    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-credential-socket\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.595037 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.595014    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c303423d-4fd4-4871-a852-acae8fb0e1f8-istiod-ca-cert\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.596922 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.596890    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-envoy\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.597232 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.597194    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-podinfo\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.603074 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.603057    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-token\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.603419 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.603400    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c897f\" (UniqueName: \"kubernetes.io/projected/c303423d-4fd4-4871-a852-acae8fb0e1f8-kube-api-access-c897f\") pod \"e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") " pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.733393 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.733315    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:50.780451 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:50.780399    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:51.001935 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:51.001859    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:51.013738 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:51.013711    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"]
Sep 14 08:08:51.015460 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:08:51.015425    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc303423d_4fd4_4871_a852_acae8fb0e1f8.slice/crio-8b415a0dac6398e89a05fb0d1fc806d1b4f8ed7091d887c641f46513ee3823cd WatchSource:0}: Error finding container 8b415a0dac6398e89a05fb0d1fc806d1b4f8ed7091d887c641f46513ee3823cd: Status 404 returned error can't find the container with id 8b415a0dac6398e89a05fb0d1fc806d1b4f8ed7091d887c641f46513ee3823cd
Sep 14 08:08:51.017682 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:51.017645    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:08:51.017783 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:51.017718    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:08:51.017783 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:51.017746    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:08:51.235782 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:51.235739    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:51.357551 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:51.357517    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"]
Sep 14 08:08:51.361181 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:51.361125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:51.439766 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:51.439733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:51.741163 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:51.741079    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2" event={"ID":"c303423d-4fd4-4871-a852-acae8fb0e1f8","Type":"ContainerStarted","Data":"5efc45ead7fc41f84be1925f07ad65a7976dcab322f1075f3e7af5d300db826b"}
Sep 14 08:08:51.741163 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:51.741134    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2" event={"ID":"c303423d-4fd4-4871-a852-acae8fb0e1f8","Type":"ContainerStarted","Data":"8b415a0dac6398e89a05fb0d1fc806d1b4f8ed7091d887c641f46513ee3823cd"}
Sep 14 08:08:51.762445 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:51.762391    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2" podStartSLOduration=1.762377313 podStartE2EDuration="1.762377313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:51.759865621 +0000 UTC m=+1186.464404150" watchObservedRunningTime="2026-09-14 08:08:51.762377313 +0000 UTC m=+1186.466915814"
Sep 14 08:08:52.734200 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:52.734171    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:52.739350 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:52.739327    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:52.746145 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:52.746123    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:52.747013 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:52.746995    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:08:53.000284 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:53.000165    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:53.638387 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:53.638349    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:53.730627 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:53.730594    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:54.315729 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:54.315694    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:54.538623 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:54.538570    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:55.420852 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:55.420806    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:56.162943 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:56.162881    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:56.433928 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:56.433855    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:56.831404 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:56.831333    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:57.643692 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:57.643660    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:57.717994 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:57.717967    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:58.528189 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:58.528157    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:58.629640 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:58.629606    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:58.763968 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:58.763934    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:59.432429 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:08:59.432397    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:00.172063 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:00.172029    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:00.978620 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:00.978577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:04.657575 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:04.657537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:04.675649 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:04.675607    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt" podUID="1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460" containerName="main" containerID="cri-o://bed01a35eea2bb07fbd3164d9e155ea0342e8942fd8ab2064d8b66bb4d5c174b" gracePeriod=45
Sep 14 08:09:04.813034 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:04.813006    2576 generic.go:356] "Generic (PLEG): container finished" podID="1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460" containerID="bed01a35eea2bb07fbd3164d9e155ea0342e8942fd8ab2064d8b66bb4d5c174b" exitCode=0
Sep 14 08:09:04.813179 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:04.813068    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt" event={"ID":"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460","Type":"ContainerDied","Data":"bed01a35eea2bb07fbd3164d9e155ea0342e8942fd8ab2064d8b66bb4d5c174b"}
Sep 14 08:09:04.940451 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:04.940425    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:09:05.041738 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.041706    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-tmp-dir\") pod \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") "
Sep 14 08:09:05.041887 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.041748    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4n8j\" (UniqueName: \"kubernetes.io/projected/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-kube-api-access-v4n8j\") pod \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") "
Sep 14 08:09:05.041887 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.041794    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-model-cache\") pod \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") "
Sep 14 08:09:05.041988 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.041897    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-home\") pod \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") "
Sep 14 08:09:05.041988 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.041931    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-tls-certs\") pod \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") "
Sep 14 08:09:05.041988 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.041958    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460" (UID: "1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:05.041988 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.041980    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-dshm\") pod \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\" (UID: \"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460\") "
Sep 14 08:09:05.042147 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.042127    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-home" (OuterVolumeSpecName: "home") pod "1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460" (UID: "1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:05.042192 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.042149    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-model-cache" (OuterVolumeSpecName: "model-cache") pod "1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460" (UID: "1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:05.042431 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.042406    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-tmp-dir\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:05.042431 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.042428    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-model-cache\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:05.042592 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.042440    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-home\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:05.044147 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.044117    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-dshm" (OuterVolumeSpecName: "dshm") pod "1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460" (UID: "1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:05.044259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.044167    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460" (UID: "1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:09:05.044259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.044183    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-kube-api-access-v4n8j" (OuterVolumeSpecName: "kube-api-access-v4n8j") pod "1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460" (UID: "1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460"). InnerVolumeSpecName "kube-api-access-v4n8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:05.143818 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.143793    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-tls-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:05.143818 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.143815    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-dshm\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:05.143981 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.143825    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4n8j\" (UniqueName: \"kubernetes.io/projected/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460-kube-api-access-v4n8j\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:05.202443 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.202362    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:05.273422 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.273386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:05.371154 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.371125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:05.433345 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.433315    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:05.751966 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.751938    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:05.819728 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.819702    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"
Sep 14 08:09:05.819728 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.819717    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt" event={"ID":"1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460","Type":"ContainerDied","Data":"a0a067186e35ba0337c145e0ba70f56c026b0cf45d5f4c561b716e3f7ecf3d98"}
Sep 14 08:09:05.819977 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.819772    2576 scope.go:117] "RemoveContainer" containerID="bed01a35eea2bb07fbd3164d9e155ea0342e8942fd8ab2064d8b66bb4d5c174b"
Sep 14 08:09:05.863372 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.863335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:05.863598 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.863573    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"]
Sep 14 08:09:05.866030 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:05.866006    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e7b0a8/test-model-e7b0a8-kserve-6dd9c597dc-ggnjt"]
Sep 14 08:09:06.378221 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.378172    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6" podUID="8b6babd7-f292-4b5c-8fbc-78401b81e5ad" containerName="main" containerID="cri-o://88f7aaceafc66f57b9ea93d56168371d253576202333f2cedb39a3c863590032" gracePeriod=45
Sep 14 08:09:06.551749 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.551630    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:06.635779 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.635711    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:09:06.758919 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.758892    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-tmp-dir\") pod \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") "
Sep 14 08:09:06.759308 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.758928    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-model-cache\") pod \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") "
Sep 14 08:09:06.759308 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.759009    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-home\") pod \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") "
Sep 14 08:09:06.759308 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.759044    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tp7lb\" (UniqueName: \"kubernetes.io/projected/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-kube-api-access-tp7lb\") pod \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") "
Sep 14 08:09:06.759308 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.759095    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-tls-certs\") pod \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") "
Sep 14 08:09:06.759308 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.759122    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-dshm\") pod \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\" (UID: \"8b6babd7-f292-4b5c-8fbc-78401b81e5ad\") "
Sep 14 08:09:06.759308 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.759200    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8b6babd7-f292-4b5c-8fbc-78401b81e5ad" (UID: "8b6babd7-f292-4b5c-8fbc-78401b81e5ad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:06.759308 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.759238    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-home" (OuterVolumeSpecName: "home") pod "8b6babd7-f292-4b5c-8fbc-78401b81e5ad" (UID: "8b6babd7-f292-4b5c-8fbc-78401b81e5ad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:06.759308 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.759255    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-model-cache" (OuterVolumeSpecName: "model-cache") pod "8b6babd7-f292-4b5c-8fbc-78401b81e5ad" (UID: "8b6babd7-f292-4b5c-8fbc-78401b81e5ad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:06.759668 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.759458    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-tmp-dir\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:06.759668 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.759477    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-model-cache\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:06.759668 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.759490    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-home\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:06.761374 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.761350    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-dshm" (OuterVolumeSpecName: "dshm") pod "8b6babd7-f292-4b5c-8fbc-78401b81e5ad" (UID: "8b6babd7-f292-4b5c-8fbc-78401b81e5ad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:06.761451 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.761367    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-kube-api-access-tp7lb" (OuterVolumeSpecName: "kube-api-access-tp7lb") pod "8b6babd7-f292-4b5c-8fbc-78401b81e5ad" (UID: "8b6babd7-f292-4b5c-8fbc-78401b81e5ad"). InnerVolumeSpecName "kube-api-access-tp7lb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:06.761451 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.761391    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8b6babd7-f292-4b5c-8fbc-78401b81e5ad" (UID: "8b6babd7-f292-4b5c-8fbc-78401b81e5ad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:09:06.826358 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.826336    2576 generic.go:356] "Generic (PLEG): container finished" podID="8b6babd7-f292-4b5c-8fbc-78401b81e5ad" containerID="88f7aaceafc66f57b9ea93d56168371d253576202333f2cedb39a3c863590032" exitCode=0
Sep 14 08:09:06.826468 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.826393    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"
Sep 14 08:09:06.826468 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.826420    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6" event={"ID":"8b6babd7-f292-4b5c-8fbc-78401b81e5ad","Type":"ContainerDied","Data":"88f7aaceafc66f57b9ea93d56168371d253576202333f2cedb39a3c863590032"}
Sep 14 08:09:06.826468 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.826462    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6" event={"ID":"8b6babd7-f292-4b5c-8fbc-78401b81e5ad","Type":"ContainerDied","Data":"dfa376067b7f2cd7927fd3a54ed283894fc97814feaa989dc411d5ea3a17bf20"}
Sep 14 08:09:06.826597 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.826483    2576 scope.go:117] "RemoveContainer" containerID="88f7aaceafc66f57b9ea93d56168371d253576202333f2cedb39a3c863590032"
Sep 14 08:09:06.835158 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.835144    2576 scope.go:117] "RemoveContainer" containerID="88f7aaceafc66f57b9ea93d56168371d253576202333f2cedb39a3c863590032"
Sep 14 08:09:06.835442 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:09:06.835423    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88f7aaceafc66f57b9ea93d56168371d253576202333f2cedb39a3c863590032\": container with ID starting with 88f7aaceafc66f57b9ea93d56168371d253576202333f2cedb39a3c863590032 not found: ID does not exist" containerID="88f7aaceafc66f57b9ea93d56168371d253576202333f2cedb39a3c863590032"
Sep 14 08:09:06.835497 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.835450    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88f7aaceafc66f57b9ea93d56168371d253576202333f2cedb39a3c863590032"} err="failed to get container status \"88f7aaceafc66f57b9ea93d56168371d253576202333f2cedb39a3c863590032\": rpc error: code = NotFound desc = could not find container \"88f7aaceafc66f57b9ea93d56168371d253576202333f2cedb39a3c863590032\": container with ID starting with 88f7aaceafc66f57b9ea93d56168371d253576202333f2cedb39a3c863590032 not found: ID does not exist"
Sep 14 08:09:06.847405 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.847381    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"]
Sep 14 08:09:06.850860 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.850840    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-1af772/noten-model-1af772-kserve-6f5dc96d95-vznx6"]
Sep 14 08:09:06.860496 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.860476    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tp7lb\" (UniqueName: \"kubernetes.io/projected/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-kube-api-access-tp7lb\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:06.860570 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.860498    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-tls-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:06.860570 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:06.860508    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b6babd7-f292-4b5c-8fbc-78401b81e5ad-dshm\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:07.224731 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:07.224697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:07.838137 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:07.838105    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460" path="/var/lib/kubelet/pods/1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460/volumes"
Sep 14 08:09:07.838580 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:07.838565    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b6babd7-f292-4b5c-8fbc-78401b81e5ad" path="/var/lib/kubelet/pods/8b6babd7-f292-4b5c-8fbc-78401b81e5ad/volumes"
Sep 14 08:09:07.863659 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:07.863629    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:08.643345 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:08.643313    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:09.290240 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:09.290190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:10.445838 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:10.445809    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:11.350470 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:11.350438    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:11.847780 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:11.847747    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:11.951044 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:11.951012    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:12.111396 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.111356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:12.177859 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.177829    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"]
Sep 14 08:09:12.178062 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.178041    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy" containerID="cri-o://15d8eac17ce3e4923cabb2e6bf75c93a127b899f25299c7f2a1e3565f54b9598" gracePeriod=30
Sep 14 08:09:12.239592 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.239556    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:12.490550 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.490475    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp"]
Sep 14 08:09:12.490872 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.490860    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460" containerName="main"
Sep 14 08:09:12.490915 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.490873    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460" containerName="main"
Sep 14 08:09:12.490915 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.490897    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b6babd7-f292-4b5c-8fbc-78401b81e5ad" containerName="main"
Sep 14 08:09:12.490915 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.490903    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b6babd7-f292-4b5c-8fbc-78401b81e5ad" containerName="main"
Sep 14 08:09:12.491010 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.490977    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ae8ac9a-9ec8-4f08-a0c1-1b5ce02ca460" containerName="main"
Sep 14 08:09:12.491010 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.490987    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b6babd7-f292-4b5c-8fbc-78401b81e5ad" containerName="main"
Sep 14 08:09:12.495460 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.495442    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp"
Sep 14 08:09:12.497907 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.497886    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:09:12.500356 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.500326    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp"]
Sep 14 08:09:12.617086 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.617053    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f76ba44b-5f01-48b6-a565-3efa2638f312-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp\" (UID: \"f76ba44b-5f01-48b6-a565-3efa2638f312\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp"
Sep 14 08:09:12.718248 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.718195    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f76ba44b-5f01-48b6-a565-3efa2638f312-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp\" (UID: \"f76ba44b-5f01-48b6-a565-3efa2638f312\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp"
Sep 14 08:09:12.718813 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.718790    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f76ba44b-5f01-48b6-a565-3efa2638f312-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp\" (UID: \"f76ba44b-5f01-48b6-a565-3efa2638f312\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp"
Sep 14 08:09:12.807900 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.807827    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp"
Sep 14 08:09:12.944882 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:12.944842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:13.079521 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:13.079496    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp"]
Sep 14 08:09:13.081328 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:09:13.081301    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf76ba44b_5f01_48b6_a565_3efa2638f312.slice/crio-9b6b768e2a7718a0f1dd300d807eac7f2420cfe0ce23fbc2dee576e91b2cb455 WatchSource:0}: Error finding container 9b6b768e2a7718a0f1dd300d807eac7f2420cfe0ce23fbc2dee576e91b2cb455: Status 404 returned error can't find the container with id 9b6b768e2a7718a0f1dd300d807eac7f2420cfe0ce23fbc2dee576e91b2cb455
Sep 14 08:09:13.475702 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:13.475661    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:13.738013 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:13.737985    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:13.865759 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:13.865726    2576 generic.go:356] "Generic (PLEG): container finished" podID="f76ba44b-5f01-48b6-a565-3efa2638f312" containerID="895b7be6acd6dc2b16568c9a7bb6a39c3199eba258ebb8eaf3ea23b5fa6b64ae" exitCode=0
Sep 14 08:09:13.865958 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:13.865799    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp" event={"ID":"f76ba44b-5f01-48b6-a565-3efa2638f312","Type":"ContainerDied","Data":"895b7be6acd6dc2b16568c9a7bb6a39c3199eba258ebb8eaf3ea23b5fa6b64ae"}
Sep 14 08:09:13.865958 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:13.865843    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp" event={"ID":"f76ba44b-5f01-48b6-a565-3efa2638f312","Type":"ContainerStarted","Data":"9b6b768e2a7718a0f1dd300d807eac7f2420cfe0ce23fbc2dee576e91b2cb455"}
Sep 14 08:09:14.507833 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:14.507801    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:14.509260 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:14.509232    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:09:15.009758 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:15.009734    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp"
Sep 14 08:09:15.145746 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:15.145681    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f76ba44b-5f01-48b6-a565-3efa2638f312-maas-api-service-ca\") pod \"f76ba44b-5f01-48b6-a565-3efa2638f312\" (UID: \"f76ba44b-5f01-48b6-a565-3efa2638f312\") "
Sep 14 08:09:15.145998 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:15.145971    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f76ba44b-5f01-48b6-a565-3efa2638f312-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f76ba44b-5f01-48b6-a565-3efa2638f312" (UID: "f76ba44b-5f01-48b6-a565-3efa2638f312"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:09:15.209505 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:15.209478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:15.246797 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:15.246762    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f76ba44b-5f01-48b6-a565-3efa2638f312-maas-api-service-ca\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:15.877391 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:15.877365    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp"
Sep 14 08:09:15.877714 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:15.877365    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp" event={"ID":"f76ba44b-5f01-48b6-a565-3efa2638f312","Type":"ContainerDied","Data":"9b6b768e2a7718a0f1dd300d807eac7f2420cfe0ce23fbc2dee576e91b2cb455"}
Sep 14 08:09:15.877714 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:15.877484    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b6b768e2a7718a0f1dd300d807eac7f2420cfe0ce23fbc2dee576e91b2cb455"
Sep 14 08:09:15.950767 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:15.950735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:16.577895 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:16.577858    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:17.483001 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.482974    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:09:17.571472 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.571440    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-token\") pod \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") "
Sep 14 08:09:17.571661 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.571507    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-envoy\") pod \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") "
Sep 14 08:09:17.571661 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.571560    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgxmv\" (UniqueName: \"kubernetes.io/projected/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-kube-api-access-pgxmv\") pod \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") "
Sep 14 08:09:17.571661 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.571602    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-data\") pod \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") "
Sep 14 08:09:17.571769 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.571672    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istiod-ca-cert\") pod \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") "
Sep 14 08:09:17.571769 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.571703    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-credential-socket\") pod \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") "
Sep 14 08:09:17.571769 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.571744    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-podinfo\") pod \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") "
Sep 14 08:09:17.571862 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.571782    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-workload-socket\") pod \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") "
Sep 14 08:09:17.571862 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.571854    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-workload-certs\") pod \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\" (UID: \"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08\") "
Sep 14 08:09:17.571997 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.571952    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-data" (OuterVolumeSpecName: "istio-data") pod "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" (UID: "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:17.572084 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.572043    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" (UID: "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:17.572411 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.572385    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" (UID: "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:17.572659 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.572423    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" (UID: "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:09:17.572832 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.572437    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:17.572832 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.572807    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:17.572832 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.572539    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" (UID: "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:17.573699 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.573681    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-token" (OuterVolumeSpecName: "istio-token") pod "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" (UID: "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:17.574021 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.574002    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" (UID: "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:09:17.574168 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.574144    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" (UID: "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:17.574337 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.574318    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-kube-api-access-pgxmv" (OuterVolumeSpecName: "kube-api-access-pgxmv") pod "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" (UID: "5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08"). InnerVolumeSpecName "kube-api-access-pgxmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:17.674107 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.674069    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:17.674107 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.674098    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:17.674107 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.674108    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgxmv\" (UniqueName: \"kubernetes.io/projected/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-kube-api-access-pgxmv\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:17.674107 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.674117    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:17.674400 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.674125    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:17.674400 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.674135    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:17.674400 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.674145    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:17.889164 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.889131    2576 generic.go:356] "Generic (PLEG): container finished" podID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerID="15d8eac17ce3e4923cabb2e6bf75c93a127b899f25299c7f2a1e3565f54b9598" exitCode=0
Sep 14 08:09:17.889363 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.889181    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"
Sep 14 08:09:17.889363 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.889242    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" event={"ID":"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08","Type":"ContainerDied","Data":"15d8eac17ce3e4923cabb2e6bf75c93a127b899f25299c7f2a1e3565f54b9598"}
Sep 14 08:09:17.889363 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.889294    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92" event={"ID":"5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08","Type":"ContainerDied","Data":"ea65ef8113977ec4b75231e29d2cf4c644bf43b9da58ecdb288484d14e7a3067"}
Sep 14 08:09:17.889363 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.889324    2576 scope.go:117] "RemoveContainer" containerID="15d8eac17ce3e4923cabb2e6bf75c93a127b899f25299c7f2a1e3565f54b9598"
Sep 14 08:09:17.901988 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.901968    2576 scope.go:117] "RemoveContainer" containerID="92864bf89aeaf9ebf2fbed8f3e7f51e673f50d3d94ab10298e49a6e2273b8fcd"
Sep 14 08:09:17.907597 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.907569    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"]
Sep 14 08:09:17.910896 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.910875    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e7b0a8-openshift-default-78fcfb7d9b-zxt92"]
Sep 14 08:09:17.911127 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.911116    2576 scope.go:117] "RemoveContainer" containerID="15d8eac17ce3e4923cabb2e6bf75c93a127b899f25299c7f2a1e3565f54b9598"
Sep 14 08:09:17.911378 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:09:17.911359    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15d8eac17ce3e4923cabb2e6bf75c93a127b899f25299c7f2a1e3565f54b9598\": container with ID starting with 15d8eac17ce3e4923cabb2e6bf75c93a127b899f25299c7f2a1e3565f54b9598 not found: ID does not exist" containerID="15d8eac17ce3e4923cabb2e6bf75c93a127b899f25299c7f2a1e3565f54b9598"
Sep 14 08:09:17.911461 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.911385    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15d8eac17ce3e4923cabb2e6bf75c93a127b899f25299c7f2a1e3565f54b9598"} err="failed to get container status \"15d8eac17ce3e4923cabb2e6bf75c93a127b899f25299c7f2a1e3565f54b9598\": rpc error: code = NotFound desc = could not find container \"15d8eac17ce3e4923cabb2e6bf75c93a127b899f25299c7f2a1e3565f54b9598\": container with ID starting with 15d8eac17ce3e4923cabb2e6bf75c93a127b899f25299c7f2a1e3565f54b9598 not found: ID does not exist"
Sep 14 08:09:17.911461 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.911401    2576 scope.go:117] "RemoveContainer" containerID="92864bf89aeaf9ebf2fbed8f3e7f51e673f50d3d94ab10298e49a6e2273b8fcd"
Sep 14 08:09:17.911668 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:09:17.911649    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92864bf89aeaf9ebf2fbed8f3e7f51e673f50d3d94ab10298e49a6e2273b8fcd\": container with ID starting with 92864bf89aeaf9ebf2fbed8f3e7f51e673f50d3d94ab10298e49a6e2273b8fcd not found: ID does not exist" containerID="92864bf89aeaf9ebf2fbed8f3e7f51e673f50d3d94ab10298e49a6e2273b8fcd"
Sep 14 08:09:17.911719 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:17.911678    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92864bf89aeaf9ebf2fbed8f3e7f51e673f50d3d94ab10298e49a6e2273b8fcd"} err="failed to get container status \"92864bf89aeaf9ebf2fbed8f3e7f51e673f50d3d94ab10298e49a6e2273b8fcd\": rpc error: code = NotFound desc = could not find container \"92864bf89aeaf9ebf2fbed8f3e7f51e673f50d3d94ab10298e49a6e2273b8fcd\": container with ID starting with 92864bf89aeaf9ebf2fbed8f3e7f51e673f50d3d94ab10298e49a6e2273b8fcd not found: ID does not exist"
Sep 14 08:09:18.741128 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.741091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:18.826565 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.826531    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7749-4c2f08"]
Sep 14 08:09:18.827385 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.827359    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy"
Sep 14 08:09:18.827531 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.827518    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy"
Sep 14 08:09:18.827654 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.827641    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy"
Sep 14 08:09:18.827738 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.827728    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy"
Sep 14 08:09:18.827832 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.827821    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f76ba44b-5f01-48b6-a565-3efa2638f312" containerName="revoke-keys"
Sep 14 08:09:18.827915 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.827905    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f76ba44b-5f01-48b6-a565-3efa2638f312" containerName="revoke-keys"
Sep 14 08:09:18.828101 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.828087    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f76ba44b-5f01-48b6-a565-3efa2638f312" containerName="revoke-keys"
Sep 14 08:09:18.828233 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.828219    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy"
Sep 14 08:09:18.828336 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.828325    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" containerName="istio-proxy"
Sep 14 08:09:18.835593 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.833477    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-4c2f08"
Sep 14 08:09:18.835961 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.835943    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qcf9c\""
Sep 14 08:09:18.837066 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.837037    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-4c2f08"]
Sep 14 08:09:18.988090 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:18.988060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9xjt\" (UniqueName: \"kubernetes.io/projected/c0e3883d-5c6a-4867-857f-eb309c8d3211-kube-api-access-j9xjt\") pod \"test-curl-7749-4c2f08\" (UID: \"c0e3883d-5c6a-4867-857f-eb309c8d3211\") " pod="opendatahub/test-curl-7749-4c2f08"
Sep 14 08:09:19.089241 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:19.089143    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9xjt\" (UniqueName: \"kubernetes.io/projected/c0e3883d-5c6a-4867-857f-eb309c8d3211-kube-api-access-j9xjt\") pod \"test-curl-7749-4c2f08\" (UID: \"c0e3883d-5c6a-4867-857f-eb309c8d3211\") " pod="opendatahub/test-curl-7749-4c2f08"
Sep 14 08:09:19.096201 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:19.096170    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9xjt\" (UniqueName: \"kubernetes.io/projected/c0e3883d-5c6a-4867-857f-eb309c8d3211-kube-api-access-j9xjt\") pod \"test-curl-7749-4c2f08\" (UID: \"c0e3883d-5c6a-4867-857f-eb309c8d3211\") " pod="opendatahub/test-curl-7749-4c2f08"
Sep 14 08:09:19.150959 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:19.150925    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-4c2f08"
Sep 14 08:09:19.422713 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:19.422398    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-4c2f08"]
Sep 14 08:09:19.425556 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:09:19.425502    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0e3883d_5c6a_4867_857f_eb309c8d3211.slice/crio-489637ba7566f14b14a050e060bfce45542bb8e6390443bbf1bb96b3af0260ef WatchSource:0}: Error finding container 489637ba7566f14b14a050e060bfce45542bb8e6390443bbf1bb96b3af0260ef: Status 404 returned error can't find the container with id 489637ba7566f14b14a050e060bfce45542bb8e6390443bbf1bb96b3af0260ef
Sep 14 08:09:19.853017 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:19.852918    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08" path="/var/lib/kubelet/pods/5bc1f23c-a74a-49e9-8a52-5fbd6e9fdf08/volumes"
Sep 14 08:09:19.900978 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:19.900948    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-4c2f08" event={"ID":"c0e3883d-5c6a-4867-857f-eb309c8d3211","Type":"ContainerStarted","Data":"489637ba7566f14b14a050e060bfce45542bb8e6390443bbf1bb96b3af0260ef"}
Sep 14 08:09:21.915366 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:21.915332    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-4c2f08" event={"ID":"c0e3883d-5c6a-4867-857f-eb309c8d3211","Type":"ContainerStarted","Data":"6f3d629a86644d8f4191e8409b96e2705c5119ab6e6525b40f07738b5084ed77"}
Sep 14 08:09:21.929713 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:21.929664    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7749-4c2f08" podStartSLOduration=1.565902043 podStartE2EDuration="3.929649219s" totalImagesPullingTime="2.363747176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:21.928013715 +0000 UTC m=+1216.632552234" watchObservedRunningTime="2026-09-14 08:09:21.929649219 +0000 UTC m=+1216.634187720"
Sep 14 08:09:22.520283 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:22.520242    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7749-4c2f08"]
Sep 14 08:09:22.524595 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:22.524557    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7749-4c2f08"]
Sep 14 08:09:22.614100 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:22.614071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:22.634930 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:22.634894    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7749-24c84c"]
Sep 14 08:09:22.635393 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:22.635373    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0e3883d-5c6a-4867-857f-eb309c8d3211" containerName="test-curl-7749-4c2f08"
Sep 14 08:09:22.635393 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:22.635394    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0e3883d-5c6a-4867-857f-eb309c8d3211" containerName="test-curl-7749-4c2f08"
Sep 14 08:09:22.635549 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:22.635522    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0e3883d-5c6a-4867-857f-eb309c8d3211" containerName="test-curl-7749-4c2f08"
Sep 14 08:09:22.638899 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:22.638879    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-24c84c"
Sep 14 08:09:22.644650 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:22.644622    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-24c84c"]
Sep 14 08:09:22.726014 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:22.725975    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbrgs\" (UniqueName: \"kubernetes.io/projected/d0453416-66de-4077-9108-960c625ece53-kube-api-access-hbrgs\") pod \"test-curl-7749-24c84c\" (UID: \"d0453416-66de-4077-9108-960c625ece53\") " pod="opendatahub/test-curl-7749-24c84c"
Sep 14 08:09:22.827058 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:22.826971    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbrgs\" (UniqueName: \"kubernetes.io/projected/d0453416-66de-4077-9108-960c625ece53-kube-api-access-hbrgs\") pod \"test-curl-7749-24c84c\" (UID: \"d0453416-66de-4077-9108-960c625ece53\") " pod="opendatahub/test-curl-7749-24c84c"
Sep 14 08:09:22.834638 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:22.834609    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbrgs\" (UniqueName: \"kubernetes.io/projected/d0453416-66de-4077-9108-960c625ece53-kube-api-access-hbrgs\") pod \"test-curl-7749-24c84c\" (UID: \"d0453416-66de-4077-9108-960c625ece53\") " pod="opendatahub/test-curl-7749-24c84c"
Sep 14 08:09:22.921583 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:09:22.921551    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7749-4c2f08" container="test-curl-7749-4c2f08"
Sep 14 08:09:22.952284 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:22.952251    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-24c84c"
Sep 14 08:09:23.126229 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.124748    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"]
Sep 14 08:09:23.126229 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.124988    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr" podUID="7950abdb-1459-4819-9165-545bb8752895" containerName="maas-api" containerID="cri-o://6f783d4143101ce07b4317cb2b0a4adce59f05aefd2f8dd776e134e8714dc236" gracePeriod=30
Sep 14 08:09:23.150890 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.150827    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:23.217735 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.217701    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:23.228062 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.228011    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-24c84c"]
Sep 14 08:09:23.229308 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:09:23.229279    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0453416_66de_4077_9108_960c625ece53.slice/crio-2beb654064d76c5cd6c800dba8c64e6b363ae3c2fcd215bc1cdf60e3ec4a2306 WatchSource:0}: Error finding container 2beb654064d76c5cd6c800dba8c64e6b363ae3c2fcd215bc1cdf60e3ec4a2306: Status 404 returned error can't find the container with id 2beb654064d76c5cd6c800dba8c64e6b363ae3c2fcd215bc1cdf60e3ec4a2306
Sep 14 08:09:23.317398 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.317369    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:23.330197 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.330154    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp"]
Sep 14 08:09:23.334016 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.330743    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f67e60-l52bp"]
Sep 14 08:09:23.373476 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.373442    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:23.378634 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.378579    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:09:23.441083 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.441048    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7950abdb-1459-4819-9165-545bb8752895-openshift-service-ca\") pod \"7950abdb-1459-4819-9165-545bb8752895\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") "
Sep 14 08:09:23.441746 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.441724    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-metrics-certs\") pod \"7950abdb-1459-4819-9165-545bb8752895\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") "
Sep 14 08:09:23.441919 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.441903    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-maas-api-tls\") pod \"7950abdb-1459-4819-9165-545bb8752895\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") "
Sep 14 08:09:23.442428 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.441657    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7950abdb-1459-4819-9165-545bb8752895-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7950abdb-1459-4819-9165-545bb8752895" (UID: "7950abdb-1459-4819-9165-545bb8752895"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:09:23.442673 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.442650    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgbsn\" (UniqueName: \"kubernetes.io/projected/7950abdb-1459-4819-9165-545bb8752895-kube-api-access-hgbsn\") pod \"7950abdb-1459-4819-9165-545bb8752895\" (UID: \"7950abdb-1459-4819-9165-545bb8752895\") "
Sep 14 08:09:23.443184 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.443164    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7950abdb-1459-4819-9165-545bb8752895-openshift-service-ca\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:23.446852 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.446827    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7950abdb-1459-4819-9165-545bb8752895-kube-api-access-hgbsn" (OuterVolumeSpecName: "kube-api-access-hgbsn") pod "7950abdb-1459-4819-9165-545bb8752895" (UID: "7950abdb-1459-4819-9165-545bb8752895"). InnerVolumeSpecName "kube-api-access-hgbsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:23.447960 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.447937    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7950abdb-1459-4819-9165-545bb8752895" (UID: "7950abdb-1459-4819-9165-545bb8752895"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:09:23.448453 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.448314    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7950abdb-1459-4819-9165-545bb8752895" (UID: "7950abdb-1459-4819-9165-545bb8752895"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:09:23.544569 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.544532    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgbsn\" (UniqueName: \"kubernetes.io/projected/7950abdb-1459-4819-9165-545bb8752895-kube-api-access-hgbsn\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:23.544569 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.544563    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-metrics-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:23.544569 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.544576    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7950abdb-1459-4819-9165-545bb8752895-maas-api-tls\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:23.686959 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.686878    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:23.746991 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.746958    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 14 08:09:23.747117 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.747019    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2" podUID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 14 08:09:23.848131 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.848101    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f76ba44b-5f01-48b6-a565-3efa2638f312" path="/var/lib/kubelet/pods/f76ba44b-5f01-48b6-a565-3efa2638f312/volumes"
Sep 14 08:09:23.929154 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.929112    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-24c84c" event={"ID":"d0453416-66de-4077-9108-960c625ece53","Type":"ContainerStarted","Data":"5ef7af3c83a7677d1f6e41dc2005b93aa7373fa3ffd1219dd20c6b153a8c1657"}
Sep 14 08:09:23.929590 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.929166    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-24c84c" event={"ID":"d0453416-66de-4077-9108-960c625ece53","Type":"ContainerStarted","Data":"2beb654064d76c5cd6c800dba8c64e6b363ae3c2fcd215bc1cdf60e3ec4a2306"}
Sep 14 08:09:23.930530 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.930503    2576 generic.go:356] "Generic (PLEG): container finished" podID="7950abdb-1459-4819-9165-545bb8752895" containerID="6f783d4143101ce07b4317cb2b0a4adce59f05aefd2f8dd776e134e8714dc236" exitCode=0
Sep 14 08:09:23.930627 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.930540    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"
Sep 14 08:09:23.930627 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.930582    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr" event={"ID":"7950abdb-1459-4819-9165-545bb8752895","Type":"ContainerDied","Data":"6f783d4143101ce07b4317cb2b0a4adce59f05aefd2f8dd776e134e8714dc236"}
Sep 14 08:09:23.930627 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.930627    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr" event={"ID":"7950abdb-1459-4819-9165-545bb8752895","Type":"ContainerDied","Data":"48539be10666df17405a420f13618ac4c0902d6ba758e5c0108b4bcc94fcf813"}
Sep 14 08:09:23.930732 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.930653    2576 scope.go:117] "RemoveContainer" containerID="6f783d4143101ce07b4317cb2b0a4adce59f05aefd2f8dd776e134e8714dc236"
Sep 14 08:09:23.930851 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.930829    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7749-4c2f08" podUID="c0e3883d-5c6a-4867-857f-eb309c8d3211" containerName="test-curl-7749-4c2f08" containerID="cri-o://6f3d629a86644d8f4191e8409b96e2705c5119ab6e6525b40f07738b5084ed77" gracePeriod=2
Sep 14 08:09:23.940537 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.940520    2576 scope.go:117] "RemoveContainer" containerID="6f783d4143101ce07b4317cb2b0a4adce59f05aefd2f8dd776e134e8714dc236"
Sep 14 08:09:23.940813 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:09:23.940788    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f783d4143101ce07b4317cb2b0a4adce59f05aefd2f8dd776e134e8714dc236\": container with ID starting with 6f783d4143101ce07b4317cb2b0a4adce59f05aefd2f8dd776e134e8714dc236 not found: ID does not exist" containerID="6f783d4143101ce07b4317cb2b0a4adce59f05aefd2f8dd776e134e8714dc236"
Sep 14 08:09:23.940862 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.940817    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f783d4143101ce07b4317cb2b0a4adce59f05aefd2f8dd776e134e8714dc236"} err="failed to get container status \"6f783d4143101ce07b4317cb2b0a4adce59f05aefd2f8dd776e134e8714dc236\": rpc error: code = NotFound desc = could not find container \"6f783d4143101ce07b4317cb2b0a4adce59f05aefd2f8dd776e134e8714dc236\": container with ID starting with 6f783d4143101ce07b4317cb2b0a4adce59f05aefd2f8dd776e134e8714dc236 not found: ID does not exist"
Sep 14 08:09:23.944746 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.944707    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7749-24c84c" podStartSLOduration=1.9446941629999999 podStartE2EDuration="1.944694163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:23.94239275 +0000 UTC m=+1218.646931254" watchObservedRunningTime="2026-09-14 08:09:23.944694163 +0000 UTC m=+1218.649232713"
Sep 14 08:09:23.954450 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.954424    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"]
Sep 14 08:09:23.959633 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:23.959607    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f67e60-5584dd6bc4-vv9xr"]
Sep 14 08:09:24.433634 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:24.433599    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:24.531538 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:24.531495    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:24.532293 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:24.532268    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7749-24c84c"]
Sep 14 08:09:24.534969 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:24.534948    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7749-24c84c"]
Sep 14 08:09:24.632442 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:24.632406    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:24.937442 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:09:24.937418    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7749-24c84c" container="test-curl-7749-24c84c"
Sep 14 08:09:25.112737 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.112709    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:25.212498 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.212405    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:25.369124 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.369093    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"]
Sep 14 08:09:25.369549 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.369534    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0453416-66de-4077-9108-960c625ece53" containerName="test-curl-7749-24c84c"
Sep 14 08:09:25.369613 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.369552    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0453416-66de-4077-9108-960c625ece53" containerName="test-curl-7749-24c84c"
Sep 14 08:09:25.369613 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.369583    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7950abdb-1459-4819-9165-545bb8752895" containerName="maas-api"
Sep 14 08:09:25.369613 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.369589    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7950abdb-1459-4819-9165-545bb8752895" containerName="maas-api"
Sep 14 08:09:25.369713 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.369656    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0453416-66de-4077-9108-960c625ece53" containerName="test-curl-7749-24c84c"
Sep 14 08:09:25.369713 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.369664    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7950abdb-1459-4819-9165-545bb8752895" containerName="maas-api"
Sep 14 08:09:25.371952 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.371934    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.374692 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.374598    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2a94f0ce-openshift-default-dockercfg-7hwjr\""
Sep 14 08:09:25.388315 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.388289    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"]
Sep 14 08:09:25.409384 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.409357    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:25.467675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.464828    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-credential-socket\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.467675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.464878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-workload-certs\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.467675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.464963    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-token\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.467675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.464989    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e10e6ba3-55bf-46de-8254-fa798815b1b2-istiod-ca-cert\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.467675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.465019    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9fsw\" (UniqueName: \"kubernetes.io/projected/e10e6ba3-55bf-46de-8254-fa798815b1b2-kube-api-access-q9fsw\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.467675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.465065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-workload-socket\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.467675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.465089    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-data\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.467675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.465135    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-podinfo\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.467675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.465226    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-envoy\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.566420 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.566386    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-credential-socket\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.566420 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.566422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-workload-certs\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.566657 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.566469    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-token\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.566657 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.566580    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e10e6ba3-55bf-46de-8254-fa798815b1b2-istiod-ca-cert\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.566657 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.566629    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9fsw\" (UniqueName: \"kubernetes.io/projected/e10e6ba3-55bf-46de-8254-fa798815b1b2-kube-api-access-q9fsw\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.566831 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.566679    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-workload-socket\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.566831 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.566710    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-data\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.566831 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.566756    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-podinfo\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.566831 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.566824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-envoy\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.567034 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.566857    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-credential-socket\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.567034 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.566900    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-workload-certs\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.567034 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.567008    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-workload-socket\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.567153 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.567114    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-data\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.567337 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.567316    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e10e6ba3-55bf-46de-8254-fa798815b1b2-istiod-ca-cert\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.569123 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.569099    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-envoy\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.569311 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.569294    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-podinfo\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.578592 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.578570    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9fsw\" (UniqueName: \"kubernetes.io/projected/e10e6ba3-55bf-46de-8254-fa798815b1b2-kube-api-access-q9fsw\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.578833 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.578814    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-token\") pod \"e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") " pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.659804 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.659777    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:25.683307 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.683281    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:25.839930 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.839903    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7950abdb-1459-4819-9165-545bb8752895" path="/var/lib/kubelet/pods/7950abdb-1459-4819-9165-545bb8752895/volumes"
Sep 14 08:09:25.945434 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.945404    2576 generic.go:356] "Generic (PLEG): container finished" podID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerID="5efc45ead7fc41f84be1925f07ad65a7976dcab322f1075f3e7af5d300db826b" exitCode=0
Sep 14 08:09:25.945809 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.945481    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2" event={"ID":"c303423d-4fd4-4871-a852-acae8fb0e1f8","Type":"ContainerDied","Data":"5efc45ead7fc41f84be1925f07ad65a7976dcab322f1075f3e7af5d300db826b"}
Sep 14 08:09:25.945809 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.945595    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7749-24c84c" podUID="d0453416-66de-4077-9108-960c625ece53" containerName="test-curl-7749-24c84c" containerID="cri-o://5ef7af3c83a7677d1f6e41dc2005b93aa7373fa3ffd1219dd20c6b153a8c1657" gracePeriod=2
Sep 14 08:09:25.946062 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.946044    2576 scope.go:117] "RemoveContainer" containerID="5efc45ead7fc41f84be1925f07ad65a7976dcab322f1075f3e7af5d300db826b"
Sep 14 08:09:25.947460 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.947429    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:09:25.947547 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.947514    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:09:25.947586 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.947545    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:09:25.969161 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.969132    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"]
Sep 14 08:09:25.996054 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:09:25.996010    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode10e6ba3_55bf_46de_8254_fa798815b1b2.slice/crio-9c7b23196843c6054e22af037f689530bffd0e0a37f8e2d40f63e6a7449d164f WatchSource:0}: Error finding container 9c7b23196843c6054e22af037f689530bffd0e0a37f8e2d40f63e6a7449d164f: Status 404 returned error can't find the container with id 9c7b23196843c6054e22af037f689530bffd0e0a37f8e2d40f63e6a7449d164f
Sep 14 08:09:25.999712 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.999052    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:09:25.999712 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.999147    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:09:25.999712 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:25.999191    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:09:26.122396 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.122190    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-4c2f08"
Sep 14 08:09:26.175004 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.174975    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9xjt\" (UniqueName: \"kubernetes.io/projected/c0e3883d-5c6a-4867-857f-eb309c8d3211-kube-api-access-j9xjt\") pod \"c0e3883d-5c6a-4867-857f-eb309c8d3211\" (UID: \"c0e3883d-5c6a-4867-857f-eb309c8d3211\") "
Sep 14 08:09:26.178682 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.178649    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0e3883d-5c6a-4867-857f-eb309c8d3211-kube-api-access-j9xjt" (OuterVolumeSpecName: "kube-api-access-j9xjt") pod "c0e3883d-5c6a-4867-857f-eb309c8d3211" (UID: "c0e3883d-5c6a-4867-857f-eb309c8d3211"). InnerVolumeSpecName "kube-api-access-j9xjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:26.203789 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.203755    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:26.276954 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.276882    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9xjt\" (UniqueName: \"kubernetes.io/projected/c0e3883d-5c6a-4867-857f-eb309c8d3211-kube-api-access-j9xjt\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:26.669927 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.669893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:26.952254 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.952139    2576 generic.go:356] "Generic (PLEG): container finished" podID="c0e3883d-5c6a-4867-857f-eb309c8d3211" containerID="6f3d629a86644d8f4191e8409b96e2705c5119ab6e6525b40f07738b5084ed77" exitCode=137
Sep 14 08:09:26.952254 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.952225    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-4c2f08"
Sep 14 08:09:26.952254 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.952251    2576 scope.go:117] "RemoveContainer" containerID="6f3d629a86644d8f4191e8409b96e2705c5119ab6e6525b40f07738b5084ed77"
Sep 14 08:09:26.954153 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.954127    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl" event={"ID":"e10e6ba3-55bf-46de-8254-fa798815b1b2","Type":"ContainerStarted","Data":"5c718dcd8b142576709b698a0113c372da2c18b572ffee9c7a4f1a6eebe6f39f"}
Sep 14 08:09:26.954301 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.954275    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl" event={"ID":"e10e6ba3-55bf-46de-8254-fa798815b1b2","Type":"ContainerStarted","Data":"9c7b23196843c6054e22af037f689530bffd0e0a37f8e2d40f63e6a7449d164f"}
Sep 14 08:09:26.956391 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.956369    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2" event={"ID":"c303423d-4fd4-4871-a852-acae8fb0e1f8","Type":"ContainerStarted","Data":"4d6ea8b503ca48bf8db2be961870377f788c5be0a74db33ef76fff1b413bf6f1"}
Sep 14 08:09:26.961851 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.961832    2576 scope.go:117] "RemoveContainer" containerID="6f3d629a86644d8f4191e8409b96e2705c5119ab6e6525b40f07738b5084ed77"
Sep 14 08:09:26.962078 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:09:26.962061    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f3d629a86644d8f4191e8409b96e2705c5119ab6e6525b40f07738b5084ed77\": container with ID starting with 6f3d629a86644d8f4191e8409b96e2705c5119ab6e6525b40f07738b5084ed77 not found: ID does not exist" containerID="6f3d629a86644d8f4191e8409b96e2705c5119ab6e6525b40f07738b5084ed77"
Sep 14 08:09:26.962130 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.962086    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3d629a86644d8f4191e8409b96e2705c5119ab6e6525b40f07738b5084ed77"} err="failed to get container status \"6f3d629a86644d8f4191e8409b96e2705c5119ab6e6525b40f07738b5084ed77\": rpc error: code = NotFound desc = could not find container \"6f3d629a86644d8f4191e8409b96e2705c5119ab6e6525b40f07738b5084ed77\": container with ID starting with 6f3d629a86644d8f4191e8409b96e2705c5119ab6e6525b40f07738b5084ed77 not found: ID does not exist"
Sep 14 08:09:26.971934 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:26.971897    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl" podStartSLOduration=1.9718866240000001 podStartE2EDuration="1.971886624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:26.970780581 +0000 UTC m=+1221.675319079" watchObservedRunningTime="2026-09-14 08:09:26.971886624 +0000 UTC m=+1221.676425126"
Sep 14 08:09:26.989708 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:09:26.989683    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-4c2f08\" is forbidden: User \"system:node:ip-10-0-133-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-123.ec2.internal' and this object" podUID="c0e3883d-5c6a-4867-857f-eb309c8d3211" pod="opendatahub/test-curl-7749-4c2f08"
Sep 14 08:09:27.683942 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:27.683905    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:27.689029 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:27.689000    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:27.691025 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:09:27.690996    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-4c2f08\" is forbidden: User \"system:node:ip-10-0-133-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-123.ec2.internal' and this object" podUID="c0e3883d-5c6a-4867-857f-eb309c8d3211" pod="opendatahub/test-curl-7749-4c2f08"
Sep 14 08:09:27.734148 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:27.734115    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:09:27.734497 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:27.734467    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 14 08:09:27.734589 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:27.734522    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2" podUID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 14 08:09:27.843977 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:27.843941    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0e3883d-5c6a-4867-857f-eb309c8d3211" path="/var/lib/kubelet/pods/c0e3883d-5c6a-4867-857f-eb309c8d3211/volumes"
Sep 14 08:09:27.963080 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:27.963061    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:27.964054 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:27.964033    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:09:28.169868 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:28.169847    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-24c84c"
Sep 14 08:09:28.247374 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:28.247304    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:28.298455 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:28.298430    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbrgs\" (UniqueName: \"kubernetes.io/projected/d0453416-66de-4077-9108-960c625ece53-kube-api-access-hbrgs\") pod \"d0453416-66de-4077-9108-960c625ece53\" (UID: \"d0453416-66de-4077-9108-960c625ece53\") "
Sep 14 08:09:28.300491 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:28.300467    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0453416-66de-4077-9108-960c625ece53-kube-api-access-hbrgs" (OuterVolumeSpecName: "kube-api-access-hbrgs") pod "d0453416-66de-4077-9108-960c625ece53" (UID: "d0453416-66de-4077-9108-960c625ece53"). InnerVolumeSpecName "kube-api-access-hbrgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:28.400264 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:28.400203    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbrgs\" (UniqueName: \"kubernetes.io/projected/d0453416-66de-4077-9108-960c625ece53-kube-api-access-hbrgs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:28.734635 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:28.734592    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 14 08:09:28.734859 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:28.734666    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2" podUID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 14 08:09:28.862265 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:28.862230    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:28.960369 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:28.960332    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:28.969047 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:28.969021    2576 generic.go:356] "Generic (PLEG): container finished" podID="d0453416-66de-4077-9108-960c625ece53" containerID="5ef7af3c83a7677d1f6e41dc2005b93aa7373fa3ffd1219dd20c6b153a8c1657" exitCode=137
Sep 14 08:09:28.969496 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:28.969059    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-24c84c"
Sep 14 08:09:28.969496 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:28.969123    2576 scope.go:117] "RemoveContainer" containerID="5ef7af3c83a7677d1f6e41dc2005b93aa7373fa3ffd1219dd20c6b153a8c1657"
Sep 14 08:09:28.977792 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:28.977776    2576 scope.go:117] "RemoveContainer" containerID="5ef7af3c83a7677d1f6e41dc2005b93aa7373fa3ffd1219dd20c6b153a8c1657"
Sep 14 08:09:28.978339 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:09:28.978035    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ef7af3c83a7677d1f6e41dc2005b93aa7373fa3ffd1219dd20c6b153a8c1657\": container with ID starting with 5ef7af3c83a7677d1f6e41dc2005b93aa7373fa3ffd1219dd20c6b153a8c1657 not found: ID does not exist" containerID="5ef7af3c83a7677d1f6e41dc2005b93aa7373fa3ffd1219dd20c6b153a8c1657"
Sep 14 08:09:28.978339 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:28.978071    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ef7af3c83a7677d1f6e41dc2005b93aa7373fa3ffd1219dd20c6b153a8c1657"} err="failed to get container status \"5ef7af3c83a7677d1f6e41dc2005b93aa7373fa3ffd1219dd20c6b153a8c1657\": rpc error: code = NotFound desc = could not find container \"5ef7af3c83a7677d1f6e41dc2005b93aa7373fa3ffd1219dd20c6b153a8c1657\": container with ID starting with 5ef7af3c83a7677d1f6e41dc2005b93aa7373fa3ffd1219dd20c6b153a8c1657 not found: ID does not exist"
Sep 14 08:09:28.980762 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:09:28.980737    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-24c84c\" is forbidden: User \"system:node:ip-10-0-133-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-123.ec2.internal' and this object" podUID="d0453416-66de-4077-9108-960c625ece53" pod="opendatahub/test-curl-7749-24c84c"
Sep 14 08:09:29.060805 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:29.060702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:29.734652 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:29.734609    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 14 08:09:29.734850 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:29.734677    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2" podUID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 14 08:09:29.843934 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:29.843894    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0453416-66de-4077-9108-960c625ece53" path="/var/lib/kubelet/pods/d0453416-66de-4077-9108-960c625ece53/volumes"
Sep 14 08:09:29.963092 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:29.963058    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:30.617098 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:30.617069    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:30.717005 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:30.716965    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:30.734561 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:30.734519    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 14 08:09:30.734762 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:30.734585    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2" podUID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 14 08:09:30.859299 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:30.859260    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"]
Sep 14 08:09:30.860099 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:30.860064    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl" podUID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerName="istio-proxy" containerID="cri-o://df7927900110ea48cf014623e0684bdd75b5c5bffb35df42f83bad7b6d39c0ac" gracePeriod=30
Sep 14 08:09:30.870542 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:30.870474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:31.337054 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.336923    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:31.437160 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.437111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:31.707923 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.707887    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"]
Sep 14 08:09:31.720970 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.720476    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.724230 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.721553    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"]
Sep 14 08:09:31.726818 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.724495    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-919cc9-openshift-default-dockercfg-654dl\""
Sep 14 08:09:31.747434 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.746678    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:31.747434 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.746728    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 14 08:09:31.747434 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.746769    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2" podUID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 14 08:09:31.851227 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.850463    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-workload-certs\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.851227 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.850512    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31b495c9-37d7-4774-8335-9b3e04049f64-istiod-ca-cert\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.851227 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.850556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9l4k\" (UniqueName: \"kubernetes.io/projected/31b495c9-37d7-4774-8335-9b3e04049f64-kube-api-access-r9l4k\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.851227 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.850595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-istio-data\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.851227 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.850662    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31b495c9-37d7-4774-8335-9b3e04049f64-istio-podinfo\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.851227 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.850703    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-credential-socket\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.851227 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.850728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31b495c9-37d7-4774-8335-9b3e04049f64-istio-token\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.851227 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.850769    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-workload-socket\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.851227 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.850815    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-istio-envoy\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.952226 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.952177    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-istio-data\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.952413 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.952292    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31b495c9-37d7-4774-8335-9b3e04049f64-istio-podinfo\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.952413 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.952323    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-credential-socket\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.952413 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.952343    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31b495c9-37d7-4774-8335-9b3e04049f64-istio-token\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.952413 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.952369    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-workload-socket\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.952646 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.952434    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-istio-envoy\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.952646 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.952532    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-workload-certs\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.952646 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.952557    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31b495c9-37d7-4774-8335-9b3e04049f64-istiod-ca-cert\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.952646 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.952602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9l4k\" (UniqueName: \"kubernetes.io/projected/31b495c9-37d7-4774-8335-9b3e04049f64-kube-api-access-r9l4k\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.952646 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.952635    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-istio-data\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.952904 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.952699    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-credential-socket\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.952904 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.952757    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-workload-socket\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.953010 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.952965    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-workload-certs\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.953300 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.953275    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31b495c9-37d7-4774-8335-9b3e04049f64-istiod-ca-cert\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.954969 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.954948    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-istio-envoy\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.955139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.955122    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31b495c9-37d7-4774-8335-9b3e04049f64-istio-podinfo\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.961305 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.961153    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9l4k\" (UniqueName: \"kubernetes.io/projected/31b495c9-37d7-4774-8335-9b3e04049f64-kube-api-access-r9l4k\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.961950 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.961932    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31b495c9-37d7-4774-8335-9b3e04049f64-istio-token\") pod \"e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") " pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:31.983917 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:31.983884    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:32.065042 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:32.065007    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:32.085296 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:32.085267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:32.184771 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:32.184709    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:32.348866 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:32.348840    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"]
Sep 14 08:09:32.351147 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:09:32.351116    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31b495c9_37d7_4774_8335_9b3e04049f64.slice/crio-43e9a6a8bfd5f7665653779123a483e14b23399d18e5f5ca613d5489cabf1eee WatchSource:0}: Error finding container 43e9a6a8bfd5f7665653779123a483e14b23399d18e5f5ca613d5489cabf1eee: Status 404 returned error can't find the container with id 43e9a6a8bfd5f7665653779123a483e14b23399d18e5f5ca613d5489cabf1eee
Sep 14 08:09:32.353494 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:32.353449    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:09:32.353646 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:32.353536    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:09:32.353646 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:32.353574    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:09:32.722581 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:32.722545    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:32.780231 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:32.779076    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:09:32.780231 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:32.779968    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:09:32.781148 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:32.781115    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:09:32.816524 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:32.816502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:32.997509 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:32.997424    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq" event={"ID":"31b495c9-37d7-4774-8335-9b3e04049f64","Type":"ContainerStarted","Data":"04bbf7fa6beb0d61c53b2afcdfa16905d6dd59a5579ce014dcd9c279876d7722"}
Sep 14 08:09:32.997509 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:32.997483    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq" event={"ID":"31b495c9-37d7-4774-8335-9b3e04049f64","Type":"ContainerStarted","Data":"43e9a6a8bfd5f7665653779123a483e14b23399d18e5f5ca613d5489cabf1eee"}
Sep 14 08:09:33.021186 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:33.021141    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq" podStartSLOduration=2.021124058 podStartE2EDuration="2.021124058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:33.018234002 +0000 UTC m=+1227.722772500" watchObservedRunningTime="2026-09-14 08:09:33.021124058 +0000 UTC m=+1227.725662559"
Sep 14 08:09:33.066137 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:33.066104    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:33.071982 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:33.071956    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:33.860182 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:33.860124    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:34.006240 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:34.005678    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:34.006240 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:34.005989    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:09:34.092126 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:34.092091    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:09:35.135661 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:35.135600    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:35.938554 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:35.938524    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:36.027331 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.027298    2576 generic.go:356] "Generic (PLEG): container finished" podID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerID="df7927900110ea48cf014623e0684bdd75b5c5bffb35df42f83bad7b6d39c0ac" exitCode=0
Sep 14 08:09:36.027503 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.027438    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl" event={"ID":"1b3d784e-2474-4eee-8f41-55de57b117f2","Type":"ContainerDied","Data":"df7927900110ea48cf014623e0684bdd75b5c5bffb35df42f83bad7b6d39c0ac"}
Sep 14 08:09:36.027591 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.027529    2576 scope.go:117] "RemoveContainer" containerID="e6c62b3436407127df5d05e068fa37a245c97168a6866cfbe43fc156c0e73ced"
Sep 14 08:09:36.132955 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.132902    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:09:36.303677 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.303644    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqx5f\" (UniqueName: \"kubernetes.io/projected/1b3d784e-2474-4eee-8f41-55de57b117f2-kube-api-access-pqx5f\") pod \"1b3d784e-2474-4eee-8f41-55de57b117f2\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") "
Sep 14 08:09:36.304127 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.303686    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-credential-socket\") pod \"1b3d784e-2474-4eee-8f41-55de57b117f2\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") "
Sep 14 08:09:36.304127 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.303714    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b3d784e-2474-4eee-8f41-55de57b117f2-istiod-ca-cert\") pod \"1b3d784e-2474-4eee-8f41-55de57b117f2\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") "
Sep 14 08:09:36.304127 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.303761    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-envoy\") pod \"1b3d784e-2474-4eee-8f41-55de57b117f2\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") "
Sep 14 08:09:36.304127 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.303793    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-podinfo\") pod \"1b3d784e-2474-4eee-8f41-55de57b117f2\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") "
Sep 14 08:09:36.304127 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.303907    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-token\") pod \"1b3d784e-2474-4eee-8f41-55de57b117f2\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") "
Sep 14 08:09:36.304127 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.303931    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1b3d784e-2474-4eee-8f41-55de57b117f2" (UID: "1b3d784e-2474-4eee-8f41-55de57b117f2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:36.304127 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.303941    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-data\") pod \"1b3d784e-2474-4eee-8f41-55de57b117f2\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") "
Sep 14 08:09:36.304127 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.304044    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-workload-certs\") pod \"1b3d784e-2474-4eee-8f41-55de57b117f2\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") "
Sep 14 08:09:36.304127 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.304098    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-workload-socket\") pod \"1b3d784e-2474-4eee-8f41-55de57b117f2\" (UID: \"1b3d784e-2474-4eee-8f41-55de57b117f2\") "
Sep 14 08:09:36.304592 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.304167    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b3d784e-2474-4eee-8f41-55de57b117f2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1b3d784e-2474-4eee-8f41-55de57b117f2" (UID: "1b3d784e-2474-4eee-8f41-55de57b117f2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:09:36.304592 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.304184    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-data" (OuterVolumeSpecName: "istio-data") pod "1b3d784e-2474-4eee-8f41-55de57b117f2" (UID: "1b3d784e-2474-4eee-8f41-55de57b117f2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:36.304592 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.304345    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1b3d784e-2474-4eee-8f41-55de57b117f2" (UID: "1b3d784e-2474-4eee-8f41-55de57b117f2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:36.304592 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.304539    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1b3d784e-2474-4eee-8f41-55de57b117f2" (UID: "1b3d784e-2474-4eee-8f41-55de57b117f2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:36.304733 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.304593    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:36.304733 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.304612    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b3d784e-2474-4eee-8f41-55de57b117f2-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:36.304733 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.304626    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:36.304733 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.304638    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:36.305993 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.305957    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1b3d784e-2474-4eee-8f41-55de57b117f2" (UID: "1b3d784e-2474-4eee-8f41-55de57b117f2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:36.306106 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.306072    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1b3d784e-2474-4eee-8f41-55de57b117f2" (UID: "1b3d784e-2474-4eee-8f41-55de57b117f2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:09:36.306388 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.306364    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b3d784e-2474-4eee-8f41-55de57b117f2-kube-api-access-pqx5f" (OuterVolumeSpecName: "kube-api-access-pqx5f") pod "1b3d784e-2474-4eee-8f41-55de57b117f2" (UID: "1b3d784e-2474-4eee-8f41-55de57b117f2"). InnerVolumeSpecName "kube-api-access-pqx5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:36.306388 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.306369    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-token" (OuterVolumeSpecName: "istio-token") pod "1b3d784e-2474-4eee-8f41-55de57b117f2" (UID: "1b3d784e-2474-4eee-8f41-55de57b117f2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:36.406351 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.406294    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:36.406351 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.406317    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqx5f\" (UniqueName: \"kubernetes.io/projected/1b3d784e-2474-4eee-8f41-55de57b117f2-kube-api-access-pqx5f\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:36.406351 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.406327    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:36.406351 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.406335    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:36.406351 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.406343    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b3d784e-2474-4eee-8f41-55de57b117f2-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:09:36.744396 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:36.744309    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:37.034375 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:37.034290    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"
Sep 14 08:09:37.034375 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:37.034345    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl" event={"ID":"1b3d784e-2474-4eee-8f41-55de57b117f2","Type":"ContainerDied","Data":"70434b5a30c53c3d9d43aa4983469794373f570c796fc35f0044cdd6c259aaf3"}
Sep 14 08:09:37.034597 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:37.034402    2576 scope.go:117] "RemoveContainer" containerID="df7927900110ea48cf014623e0684bdd75b5c5bffb35df42f83bad7b6d39c0ac"
Sep 14 08:09:37.057190 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:37.057162    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"]
Sep 14 08:09:37.060801 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:37.060775    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-f67e60-openshift-default-75b5fcb4dd-tdlsl"]
Sep 14 08:09:37.523869 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:37.523840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:37.838754 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:37.838673    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b3d784e-2474-4eee-8f41-55de57b117f2" path="/var/lib/kubelet/pods/1b3d784e-2474-4eee-8f41-55de57b117f2/volumes"
Sep 14 08:09:38.191090 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:38.191056    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:38.857976 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:38.857942    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:39.620410 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.620367    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"]
Sep 14 08:09:39.621071 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.621047    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerName="istio-proxy"
Sep 14 08:09:39.621126 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.621078    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerName="istio-proxy"
Sep 14 08:09:39.621126 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.621094    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerName="istio-proxy"
Sep 14 08:09:39.621126 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.621103    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerName="istio-proxy"
Sep 14 08:09:39.621247 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.621238    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerName="istio-proxy"
Sep 14 08:09:39.626802 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.626773    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:39.629332 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.629301    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-919cc9-serving-cert\""
Sep 14 08:09:39.631665 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.631637    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"]
Sep 14 08:09:39.649897 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.649872    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:39.740925 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.740876    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05230cb1-58cf-4530-b293-3019188d3361-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:39.741274 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.741242    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbrfx\" (UniqueName: \"kubernetes.io/projected/05230cb1-58cf-4530-b293-3019188d3361-kube-api-access-nbrfx\") pod \"maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:39.741400 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.741296    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:39.741400 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.741377    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:39.750736 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.750705    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:39.842144 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.842114    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:39.842300 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.842282    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05230cb1-58cf-4530-b293-3019188d3361-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:39.842369 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.842336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbrfx\" (UniqueName: \"kubernetes.io/projected/05230cb1-58cf-4530-b293-3019188d3361-kube-api-access-nbrfx\") pod \"maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:39.842426 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.842367    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:39.842510 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:09:39.842494    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-serving-cert: secret "maas-api-e2e-shared-a-w3-919cc9-serving-cert" not found
Sep 14 08:09:39.842589 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:09:39.842578    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-maas-api-tls podName:05230cb1-58cf-4530-b293-3019188d3361 nodeName:}" failed. No retries permitted until 2026-09-14 08:09:40.342556509 +0000 UTC m=+1235.047095003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-maas-api-tls") pod "maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v" (UID: "05230cb1-58cf-4530-b293-3019188d3361") : secret "maas-api-e2e-shared-a-w3-919cc9-serving-cert" not found
Sep 14 08:09:39.842921 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.842902    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05230cb1-58cf-4530-b293-3019188d3361-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:39.844551 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.844528    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:39.849671 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:39.849647    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbrfx\" (UniqueName: \"kubernetes.io/projected/05230cb1-58cf-4530-b293-3019188d3361-kube-api-access-nbrfx\") pod \"maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:40.103083 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:40.102965    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:40.204392 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:40.204352    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:40.291530 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:40.291504    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:40.347421 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:40.347395    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:40.350104 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:40.350078    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:40.541395 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:40.541359    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:40.811117 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:40.811092    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"]
Sep 14 08:09:40.812571 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:09:40.812540    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05230cb1_58cf_4530_b293_3019188d3361.slice/crio-774b7160a79d8a57739b180392d25874c89cfc1e291773e565bab431be84e866 WatchSource:0}: Error finding container 774b7160a79d8a57739b180392d25874c89cfc1e291773e565bab431be84e866: Status 404 returned error can't find the container with id 774b7160a79d8a57739b180392d25874c89cfc1e291773e565bab431be84e866
Sep 14 08:09:41.058279 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:41.058245    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v" event={"ID":"05230cb1-58cf-4530-b293-3019188d3361","Type":"ContainerStarted","Data":"774b7160a79d8a57739b180392d25874c89cfc1e291773e565bab431be84e866"}
Sep 14 08:09:42.065140 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:42.065105    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v" event={"ID":"05230cb1-58cf-4530-b293-3019188d3361","Type":"ContainerStarted","Data":"e533d03502337102b4538d87f8f59dce2c91b18b302a6ed8dbc7dad46d1fa1ac"}
Sep 14 08:09:42.065566 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:42.065177    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:42.082899 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:42.082846    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v" podStartSLOduration=2.676251321 podStartE2EDuration="3.082829915s" totalImagesPullingTime="406.578594ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:42.08157899 +0000 UTC m=+1236.786117494" watchObservedRunningTime="2026-09-14 08:09:42.082829915 +0000 UTC m=+1236.787368417"
Sep 14 08:09:48.076823 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:48.076788    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:09:51.564117 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:51.564078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:52.962107 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:52.962033    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:53.521682 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:53.521650    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:54.125862 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:54.125829    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:54.739108 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:54.739075    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:55.608417 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:55.608347    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:56.438013 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:56.437912    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:57.061569 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:57.061474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:57.691876 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:57.691841    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:58.452699 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:58.452669    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:58.994790 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:58.994759    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:59.004329 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:59.004303    2576 patch_prober.go:28] interesting pod/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 14 08:09:59.004448 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:59.004365    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl" podUID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 14 08:09:59.130388 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:59.130348    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:59.492932 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:59.492862    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:09:59.832714 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:59.832642    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:59.934928 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:09:59.934866    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:00.177750 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.177721    2576 generic.go:356] "Generic (PLEG): container finished" podID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerID="5c718dcd8b142576709b698a0113c372da2c18b572ffee9c7a4f1a6eebe6f39f" exitCode=0
Sep 14 08:10:00.177930 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.177798    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl" event={"ID":"e10e6ba3-55bf-46de-8254-fa798815b1b2","Type":"ContainerDied","Data":"5c718dcd8b142576709b698a0113c372da2c18b572ffee9c7a4f1a6eebe6f39f"}
Sep 14 08:10:00.178308 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.178287    2576 scope.go:117] "RemoveContainer" containerID="5c718dcd8b142576709b698a0113c372da2c18b572ffee9c7a4f1a6eebe6f39f"
Sep 14 08:10:00.179465 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.179444    2576 generic.go:356] "Generic (PLEG): container finished" podID="31b495c9-37d7-4774-8335-9b3e04049f64" containerID="04bbf7fa6beb0d61c53b2afcdfa16905d6dd59a5579ce014dcd9c279876d7722" exitCode=0
Sep 14 08:10:00.179563 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.179490    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:10:00.179563 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.179510    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq" event={"ID":"31b495c9-37d7-4774-8335-9b3e04049f64","Type":"ContainerDied","Data":"04bbf7fa6beb0d61c53b2afcdfa16905d6dd59a5579ce014dcd9c279876d7722"}
Sep 14 08:10:00.179563 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.179559    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:10:00.179686 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.179592    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:10:00.179845 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.179831    2576 scope.go:117] "RemoveContainer" containerID="04bbf7fa6beb0d61c53b2afcdfa16905d6dd59a5579ce014dcd9c279876d7722"
Sep 14 08:10:00.180905 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.180880    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:10:00.180953 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.180940    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:10:00.180996 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.180976    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:10:00.237201 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.237117    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:00.499156 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.499013    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"]
Sep 14 08:10:00.499730 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.499698    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b3d784e-2474-4eee-8f41-55de57b117f2" containerName="istio-proxy"
Sep 14 08:10:00.502274 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.502189    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.513896 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.512329    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"]
Sep 14 08:10:00.514973 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.514950    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-fc0d6e-openshift-default-dockercfg-v7zgx\""
Sep 14 08:10:00.528160 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.527985    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:00.542230 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.541691    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df5656ab-c018-4e50-8f58-005290d96ee9-istio-podinfo\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.542230 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.541738    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-credential-socket\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.542230 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.541853    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-workload-socket\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.542230 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.541887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-istio-data\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.542230 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.541937    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df5656ab-c018-4e50-8f58-005290d96ee9-istio-token\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.542230 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.541964    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-workload-certs\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.542230 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.542004    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df5656ab-c018-4e50-8f58-005290d96ee9-istiod-ca-cert\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.542230 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.542039    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk8rx\" (UniqueName: \"kubernetes.io/projected/df5656ab-c018-4e50-8f58-005290d96ee9-kube-api-access-jk8rx\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.542230 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.542198    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-istio-envoy\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.619694 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.619650    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:00.647313 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.645978    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-istio-envoy\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.647313 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.646050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df5656ab-c018-4e50-8f58-005290d96ee9-istio-podinfo\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.647313 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.646086    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-credential-socket\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.647313 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.646177    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-workload-socket\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.647313 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.646233    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-istio-data\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.647313 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.646293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df5656ab-c018-4e50-8f58-005290d96ee9-istio-token\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.647313 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.646324    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-workload-certs\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.647313 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.646361    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df5656ab-c018-4e50-8f58-005290d96ee9-istiod-ca-cert\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.647313 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.646409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk8rx\" (UniqueName: \"kubernetes.io/projected/df5656ab-c018-4e50-8f58-005290d96ee9-kube-api-access-jk8rx\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.648350 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.648323    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-istio-data\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.661260 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.649481    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-workload-socket\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.661260 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.649980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-workload-certs\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.661260 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.650905    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-istio-envoy\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.661260 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.651315    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df5656ab-c018-4e50-8f58-005290d96ee9-istiod-ca-cert\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.661260 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.655713    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-credential-socket\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.661260 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.659471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df5656ab-c018-4e50-8f58-005290d96ee9-istio-token\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.669801 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.669700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df5656ab-c018-4e50-8f58-005290d96ee9-istio-podinfo\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.670360 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.670337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk8rx\" (UniqueName: \"kubernetes.io/projected/df5656ab-c018-4e50-8f58-005290d96ee9-kube-api-access-jk8rx\") pod \"e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") " pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.817869 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.817774    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:00.938053 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:00.937952    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:01.100547 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:01.100520    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"]
Sep 14 08:10:01.101865 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:10:01.101834    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf5656ab_c018_4e50_8f58_005290d96ee9.slice/crio-9b54065960f872a066b9c67111d5be6d3215f70e72361a652f8beec40b968c2f WatchSource:0}: Error finding container 9b54065960f872a066b9c67111d5be6d3215f70e72361a652f8beec40b968c2f: Status 404 returned error can't find the container with id 9b54065960f872a066b9c67111d5be6d3215f70e72361a652f8beec40b968c2f
Sep 14 08:10:01.104190 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:01.104150    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:10:01.104275 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:01.104248    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:10:01.104316 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:01.104280    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:10:01.187176 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:01.187147    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl" event={"ID":"e10e6ba3-55bf-46de-8254-fa798815b1b2","Type":"ContainerStarted","Data":"89e0229844d78a8e726bea6f9142009e12ee8995b8925a902ad1fb77a6f0b8f1"}
Sep 14 08:10:01.189244 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:01.189198    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq" event={"ID":"31b495c9-37d7-4774-8335-9b3e04049f64","Type":"ContainerStarted","Data":"f6d74d1940aa920efb3cbaf4ab915a6ca01b1cd0ca036555922dd429fa3b0743"}
Sep 14 08:10:01.190541 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:01.190481    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7" event={"ID":"df5656ab-c018-4e50-8f58-005290d96ee9","Type":"ContainerStarted","Data":"9b54065960f872a066b9c67111d5be6d3215f70e72361a652f8beec40b968c2f"}
Sep 14 08:10:01.471753 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:01.471710    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:01.683607 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:01.683559    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:10:01.684079 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:01.683755    2576 patch_prober.go:28] interesting pod/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 14 08:10:01.684079 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:01.683799    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl" podUID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 14 08:10:02.066305 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:02.066263    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:10:02.067481 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:02.067450    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 08:10:02.067623 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:02.067509    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq" podUID="31b495c9-37d7-4774-8335-9b3e04049f64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 08:10:02.197819 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:02.197778    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7" event={"ID":"df5656ab-c018-4e50-8f58-005290d96ee9","Type":"ContainerStarted","Data":"378d8d5d485f2ba82626f009903103d130752d5625d073a70476f28ae89a1a4e"}
Sep 14 08:10:02.218953 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:02.218906    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7" podStartSLOduration=2.2188914889999998 podStartE2EDuration="2.218891489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:10:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:10:02.217713174 +0000 UTC m=+1256.922251688" watchObservedRunningTime="2026-09-14 08:10:02.218891489 +0000 UTC m=+1256.923429991"
Sep 14 08:10:02.684544 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:02.684505    2576 patch_prober.go:28] interesting pod/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 14 08:10:02.684948 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:02.684559    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl" podUID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 14 08:10:02.818288 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:02.818258    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:02.846705 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:02.846664    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:03.066142 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:03.066033    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 08:10:03.066142 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:03.066108    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq" podUID="31b495c9-37d7-4774-8335-9b3e04049f64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 08:10:03.203963 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:03.203934    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:03.204801 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:03.204784    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:03.352957 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:03.352842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:03.684608 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:03.684571    2576 patch_prober.go:28] interesting pod/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 14 08:10:03.685003 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:03.684623    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl" podUID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 14 08:10:04.004575 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:04.004492    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:10:04.066346 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:04.066300    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 08:10:04.066613 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:04.066372    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq" podUID="31b495c9-37d7-4774-8335-9b3e04049f64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 08:10:04.134500 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:04.134337    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:04.227817 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:04.227708    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:04.684336 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:04.684304    2576 patch_prober.go:28] interesting pod/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 14 08:10:04.684509 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:04.684360    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl" podUID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 14 08:10:05.053408 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:05.053310    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:05.065639 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:05.065613    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 08:10:05.065843 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:05.065819    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq" podUID="31b495c9-37d7-4774-8335-9b3e04049f64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 08:10:05.684557 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:05.684523    2576 patch_prober.go:28] interesting pod/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 14 08:10:05.684732 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:05.684572    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl" podUID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 14 08:10:05.827642 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:05.827561    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:06.066622 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:06.066502    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 08:10:06.067100 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:06.066612    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq" podUID="31b495c9-37d7-4774-8335-9b3e04049f64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 08:10:06.448816 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:06.448763    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:06.689148 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:06.689117    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:10:06.690008 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:06.689984    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:10:06.693784 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:06.693759    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:10:07.069680 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:07.069652    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:10:07.070623 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:07.070606    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:10:07.166198 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:07.166169    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:07.820380 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:07.820345    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:08.521810 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.521548    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"]
Sep 14 08:10:08.525395 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.525371    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:08.527775 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.527753    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-fc0d6e-serving-cert\""
Sep 14 08:10:08.534967 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.534942    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"]
Sep 14 08:10:08.553120 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.553095    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:08.635966 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.635928    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af3c165c-c4e4-421f-a51d-244bfc0e39fb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:08.636333 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.636309    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-622qg\" (UniqueName: \"kubernetes.io/projected/af3c165c-c4e4-421f-a51d-244bfc0e39fb-kube-api-access-622qg\") pod \"maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:08.636621 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.636600    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:08.636777 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.636757    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:08.654382 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.654349    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:08.737879 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.737846    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-622qg\" (UniqueName: \"kubernetes.io/projected/af3c165c-c4e4-421f-a51d-244bfc0e39fb-kube-api-access-622qg\") pod \"maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:08.738024 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.737925    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:08.738024 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.737953    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:08.738115 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.738087    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af3c165c-c4e4-421f-a51d-244bfc0e39fb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:08.738159 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:10:08.738128    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-serving-cert: secret "maas-api-e2e-shared-b-w3-fc0d6e-serving-cert" not found
Sep 14 08:10:08.738245 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:10:08.738231    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-maas-api-tls podName:af3c165c-c4e4-421f-a51d-244bfc0e39fb nodeName:}" failed. No retries permitted until 2026-09-14 08:10:09.238190903 +0000 UTC m=+1263.942729383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-maas-api-tls") pod "maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm" (UID: "af3c165c-c4e4-421f-a51d-244bfc0e39fb") : secret "maas-api-e2e-shared-b-w3-fc0d6e-serving-cert" not found
Sep 14 08:10:08.738775 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.738757    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af3c165c-c4e4-421f-a51d-244bfc0e39fb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:08.740586 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.740563    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:08.745988 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.745971    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-622qg\" (UniqueName: \"kubernetes.io/projected/af3c165c-c4e4-421f-a51d-244bfc0e39fb-kube-api-access-622qg\") pod \"maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:08.879780 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:08.879753    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:09.032020 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:09.031991    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:09.124593 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:09.124554    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:09.250289 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:09.250171    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:09.265516 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:09.265484    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:09.438757 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:09.438716    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:09.712615 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:09.712589    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"]
Sep 14 08:10:09.713976 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:10:09.713951    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf3c165c_c4e4_421f_a51d_244bfc0e39fb.slice/crio-5c5793252bd8e7d2d18de0f56ff26b37e50c18090ee961e9750a745b00cf3762 WatchSource:0}: Error finding container 5c5793252bd8e7d2d18de0f56ff26b37e50c18090ee961e9750a745b00cf3762: Status 404 returned error can't find the container with id 5c5793252bd8e7d2d18de0f56ff26b37e50c18090ee961e9750a745b00cf3762
Sep 14 08:10:10.257789 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:10.257752    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm" event={"ID":"af3c165c-c4e4-421f-a51d-244bfc0e39fb","Type":"ContainerStarted","Data":"db96b9f4c2e16710423eea664e2b23b578ab944ae64a016eeff1e3f867d1dd51"}
Sep 14 08:10:10.257945 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:10.257818    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm" event={"ID":"af3c165c-c4e4-421f-a51d-244bfc0e39fb","Type":"ContainerStarted","Data":"5c5793252bd8e7d2d18de0f56ff26b37e50c18090ee961e9750a745b00cf3762"}
Sep 14 08:10:10.257945 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:10.257851    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:10.276453 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:10.276402    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm" podStartSLOduration=1.95535645 podStartE2EDuration="2.2763867s" totalImagesPullingTime="321.03025ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:10:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:10:10.273777791 +0000 UTC m=+1264.978316295" watchObservedRunningTime="2026-09-14 08:10:10.2763867 +0000 UTC m=+1264.980925201"
Sep 14 08:10:14.869718 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:14.869685    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:16.269733 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:16.269699    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:10:18.780174 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:18.780137    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:26.204898 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:26.204864    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:10:27.843350 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:27.843322    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:27.944386 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:27.944350    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:30.400203 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:30.400161    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:31.142877 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:31.142842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:31.773064 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:31.773030    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:32.427546 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:32.427506    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:33.143884 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:33.143849    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:33.778580 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:33.778553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:34.200072 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:34.200037    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:34.435475 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:34.435439    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:35.251035 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:35.251001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:37.416149 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:37.416119    2576 generic.go:356] "Generic (PLEG): container finished" podID="df5656ab-c018-4e50-8f58-005290d96ee9" containerID="378d8d5d485f2ba82626f009903103d130752d5625d073a70476f28ae89a1a4e" exitCode=0
Sep 14 08:10:37.416619 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:37.416227    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7" event={"ID":"df5656ab-c018-4e50-8f58-005290d96ee9","Type":"ContainerDied","Data":"378d8d5d485f2ba82626f009903103d130752d5625d073a70476f28ae89a1a4e"}
Sep 14 08:10:37.416691 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:37.416677    2576 scope.go:117] "RemoveContainer" containerID="378d8d5d485f2ba82626f009903103d130752d5625d073a70476f28ae89a1a4e"
Sep 14 08:10:37.417936 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:37.417900    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:10:37.418048 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:37.417988    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:10:37.418048 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:37.418034    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:10:38.425433 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:38.425387    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7" event={"ID":"df5656ab-c018-4e50-8f58-005290d96ee9","Type":"ContainerStarted","Data":"30ac2cd8c4349b20b96b4cd1bb24f90396b5f3b84d9bcd67360ed01dac7fcf32"}
Sep 14 08:10:38.818756 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:38.818660    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:38.818915 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:38.818870    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 14 08:10:38.818958 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:38.818915    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7" podUID="df5656ab-c018-4e50-8f58-005290d96ee9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 14 08:10:39.818974 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:39.818934    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 14 08:10:39.819387 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:39.818990    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7" podUID="df5656ab-c018-4e50-8f58-005290d96ee9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 14 08:10:40.819181 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:40.819139    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 14 08:10:40.819602 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:40.819241    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7" podUID="df5656ab-c018-4e50-8f58-005290d96ee9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 14 08:10:41.717595 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:41.717553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:41.818916 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:41.818879    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 14 08:10:41.819100 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:41.818946    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7" podUID="df5656ab-c018-4e50-8f58-005290d96ee9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 14 08:10:42.775036 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:42.775003    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:42.818543 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:42.818510    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 14 08:10:42.818697 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:42.818581    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7" podUID="df5656ab-c018-4e50-8f58-005290d96ee9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 14 08:10:42.875607 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:42.875577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:43.853943 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:43.853911    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:43.854430 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:43.854241    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:43.855089 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:43.855064    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:10:44.281542 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:44.281470    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"]
Sep 14 08:10:44.298688 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:44.298662    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:56.803967 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:56.798698    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:10:59.301487 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.301433    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg" podUID="a7610e87-7025-4ae2-b822-df0d0bff88cb" containerName="main" containerID="cri-o://a30c1456db5a2cfbe2b0a818a557ff65b57fa2222a1fa2230a87a7156359598a" gracePeriod=45
Sep 14 08:10:59.544528 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.544504    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:10:59.563045 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.562977    2576 generic.go:356] "Generic (PLEG): container finished" podID="a7610e87-7025-4ae2-b822-df0d0bff88cb" containerID="a30c1456db5a2cfbe2b0a818a557ff65b57fa2222a1fa2230a87a7156359598a" exitCode=0
Sep 14 08:10:59.563045 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.563026    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"
Sep 14 08:10:59.563238 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.563073    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg" event={"ID":"a7610e87-7025-4ae2-b822-df0d0bff88cb","Type":"ContainerDied","Data":"a30c1456db5a2cfbe2b0a818a557ff65b57fa2222a1fa2230a87a7156359598a"}
Sep 14 08:10:59.563238 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.563122    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg" event={"ID":"a7610e87-7025-4ae2-b822-df0d0bff88cb","Type":"ContainerDied","Data":"d4758c881ffbdd067d8f52e98aff2c73f497b766ad7946977b4bdf8a59dfb502"}
Sep 14 08:10:59.563238 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.563142    2576 scope.go:117] "RemoveContainer" containerID="a30c1456db5a2cfbe2b0a818a557ff65b57fa2222a1fa2230a87a7156359598a"
Sep 14 08:10:59.572972 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.572946    2576 scope.go:117] "RemoveContainer" containerID="a30c1456db5a2cfbe2b0a818a557ff65b57fa2222a1fa2230a87a7156359598a"
Sep 14 08:10:59.573332 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:10:59.573310    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a30c1456db5a2cfbe2b0a818a557ff65b57fa2222a1fa2230a87a7156359598a\": container with ID starting with a30c1456db5a2cfbe2b0a818a557ff65b57fa2222a1fa2230a87a7156359598a not found: ID does not exist" containerID="a30c1456db5a2cfbe2b0a818a557ff65b57fa2222a1fa2230a87a7156359598a"
Sep 14 08:10:59.573391 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.573340    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a30c1456db5a2cfbe2b0a818a557ff65b57fa2222a1fa2230a87a7156359598a"} err="failed to get container status \"a30c1456db5a2cfbe2b0a818a557ff65b57fa2222a1fa2230a87a7156359598a\": rpc error: code = NotFound desc = could not find container \"a30c1456db5a2cfbe2b0a818a557ff65b57fa2222a1fa2230a87a7156359598a\": container with ID starting with a30c1456db5a2cfbe2b0a818a557ff65b57fa2222a1fa2230a87a7156359598a not found: ID does not exist"
Sep 14 08:10:59.667533 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.667494    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-home\") pod \"a7610e87-7025-4ae2-b822-df0d0bff88cb\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") "
Sep 14 08:10:59.667735 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.667584    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7610e87-7025-4ae2-b822-df0d0bff88cb-tls-certs\") pod \"a7610e87-7025-4ae2-b822-df0d0bff88cb\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") "
Sep 14 08:10:59.667735 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.667672    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-dshm\") pod \"a7610e87-7025-4ae2-b822-df0d0bff88cb\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") "
Sep 14 08:10:59.667735 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.667689    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-model-cache\") pod \"a7610e87-7025-4ae2-b822-df0d0bff88cb\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") "
Sep 14 08:10:59.667735 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.667713    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-tmp-dir\") pod \"a7610e87-7025-4ae2-b822-df0d0bff88cb\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") "
Sep 14 08:10:59.667735 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.667734    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5gsk\" (UniqueName: \"kubernetes.io/projected/a7610e87-7025-4ae2-b822-df0d0bff88cb-kube-api-access-z5gsk\") pod \"a7610e87-7025-4ae2-b822-df0d0bff88cb\" (UID: \"a7610e87-7025-4ae2-b822-df0d0bff88cb\") "
Sep 14 08:10:59.668004 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.667815    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-home" (OuterVolumeSpecName: "home") pod "a7610e87-7025-4ae2-b822-df0d0bff88cb" (UID: "a7610e87-7025-4ae2-b822-df0d0bff88cb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:10:59.668004 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.667952    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-model-cache" (OuterVolumeSpecName: "model-cache") pod "a7610e87-7025-4ae2-b822-df0d0bff88cb" (UID: "a7610e87-7025-4ae2-b822-df0d0bff88cb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:10:59.668004 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.667975    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a7610e87-7025-4ae2-b822-df0d0bff88cb" (UID: "a7610e87-7025-4ae2-b822-df0d0bff88cb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:10:59.668193 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.668162    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-tmp-dir\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:10:59.668193 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.668190    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-home\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:10:59.668193 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.668199    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-model-cache\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:10:59.670023 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.669992    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-dshm" (OuterVolumeSpecName: "dshm") pod "a7610e87-7025-4ae2-b822-df0d0bff88cb" (UID: "a7610e87-7025-4ae2-b822-df0d0bff88cb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:10:59.670103 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.670042    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7610e87-7025-4ae2-b822-df0d0bff88cb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a7610e87-7025-4ae2-b822-df0d0bff88cb" (UID: "a7610e87-7025-4ae2-b822-df0d0bff88cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:10:59.670103 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.670052    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7610e87-7025-4ae2-b822-df0d0bff88cb-kube-api-access-z5gsk" (OuterVolumeSpecName: "kube-api-access-z5gsk") pod "a7610e87-7025-4ae2-b822-df0d0bff88cb" (UID: "a7610e87-7025-4ae2-b822-df0d0bff88cb"). InnerVolumeSpecName "kube-api-access-z5gsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:10:59.769614 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.769580    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7610e87-7025-4ae2-b822-df0d0bff88cb-tls-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:10:59.769614 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.769610    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7610e87-7025-4ae2-b822-df0d0bff88cb-dshm\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:10:59.769614 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.769618    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5gsk\" (UniqueName: \"kubernetes.io/projected/a7610e87-7025-4ae2-b822-df0d0bff88cb-kube-api-access-z5gsk\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:10:59.878867 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.878833    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"]
Sep 14 08:10:59.880914 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:10:59.880892    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/e2e-conflict-a-2827d38c-kserve-57f865d84b-zd2hg"]
Sep 14 08:11:01.378761 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.378726    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"]
Sep 14 08:11:01.379465 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.379439    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7610e87-7025-4ae2-b822-df0d0bff88cb" containerName="main"
Sep 14 08:11:01.379621 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.379609    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7610e87-7025-4ae2-b822-df0d0bff88cb" containerName="main"
Sep 14 08:11:01.379895 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.379881    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7610e87-7025-4ae2-b822-df0d0bff88cb" containerName="main"
Sep 14 08:11:01.389757 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.389730    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.392302 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.392273    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-919cc9\"/\"openshift-service-ca.crt\""
Sep 14 08:11:01.392890 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.392835    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"]
Sep 14 08:11:01.393341 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.393314    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-919cc9\"/\"auth-test-model-919cc9-kserve-self-signed-certs\""
Sep 14 08:11:01.393820 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.393717    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-919cc9\"/\"kube-root-ca.crt\""
Sep 14 08:11:01.394023 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.393998    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-919cc9\"/\"default-dockercfg-7hm7s\""
Sep 14 08:11:01.411164 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.411136    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:01.486115 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.486080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-model-cache\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.486115 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.486117    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46858da9-0ca0-4607-b32c-df0351851164-tls-certs\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.486355 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.486139    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-tmp-dir\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.486355 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.486198    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-dshm\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.486355 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.486333    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-home\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.486465 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.486402    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbq5z\" (UniqueName: \"kubernetes.io/projected/46858da9-0ca0-4607-b32c-df0351851164-kube-api-access-xbq5z\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.587448 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.587416    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-home\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.587614 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.587492    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbq5z\" (UniqueName: \"kubernetes.io/projected/46858da9-0ca0-4607-b32c-df0351851164-kube-api-access-xbq5z\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.587614 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.587517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-model-cache\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.587614 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.587535    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46858da9-0ca0-4607-b32c-df0351851164-tls-certs\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.587614 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.587559    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-tmp-dir\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.587614 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.587587    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-dshm\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.587885 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.587866    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-home\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.587941 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.587880    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-model-cache\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.588008 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.587984    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-tmp-dir\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.589902 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.589879    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-dshm\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.590088 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.590070    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46858da9-0ca0-4607-b32c-df0351851164-tls-certs\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.594929 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.594908    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbq5z\" (UniqueName: \"kubernetes.io/projected/46858da9-0ca0-4607-b32c-df0351851164-kube-api-access-xbq5z\") pod \"auth-test-model-919cc9-kserve-7b4ddd77bc-778sb\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.701261 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.701175    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:01.849822 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.849197    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7610e87-7025-4ae2-b822-df0d0bff88cb" path="/var/lib/kubelet/pods/a7610e87-7025-4ae2-b822-df0d0bff88cb/volumes"
Sep 14 08:11:01.975027 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:01.974943    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"]
Sep 14 08:11:01.978326 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:11:01.978294    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46858da9_0ca0_4607_b32c_df0351851164.slice/crio-1bd91cf676645b56b847c4b003d17e995e7bcb79240bbc3f509b1e19d2fe3e13 WatchSource:0}: Error finding container 1bd91cf676645b56b847c4b003d17e995e7bcb79240bbc3f509b1e19d2fe3e13: Status 404 returned error can't find the container with id 1bd91cf676645b56b847c4b003d17e995e7bcb79240bbc3f509b1e19d2fe3e13
Sep 14 08:11:02.582867 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:02.582834    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb" event={"ID":"46858da9-0ca0-4607-b32c-df0351851164","Type":"ContainerStarted","Data":"72ef10d6e52f48a4df90bb951576dd173ac8ae24608be58c95cde39794a343d9"}
Sep 14 08:11:02.583245 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:02.582873    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb" event={"ID":"46858da9-0ca0-4607-b32c-df0351851164","Type":"ContainerStarted","Data":"1bd91cf676645b56b847c4b003d17e995e7bcb79240bbc3f509b1e19d2fe3e13"}
Sep 14 08:11:02.599382 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:02.599338    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb" podStartSLOduration=1.59932353 podStartE2EDuration="1.59932353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:11:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:11:02.597320984 +0000 UTC m=+1317.301859499" watchObservedRunningTime="2026-09-14 08:11:02.59932353 +0000 UTC m=+1317.303862031"
Sep 14 08:11:11.702265 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:11.702229    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:11.702700 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:11.702279    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:11.714931 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:11.714903    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:12.657956 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:12.657927    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:11:23.871422 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:23.871391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:23.972009 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:23.971978    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:27.996100 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:27.996065    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:11:29.652026 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:29.651988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:39.842032 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:39.841990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:46.308893 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:46.308858    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:47.022080 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:47.022047    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:47.676088 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:47.676053    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:48.329825 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:48.329793    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:49.139149 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:49.139106    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:49.770786 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:49.770755    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:50.423577 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:50.423542    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:50.497601 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:50.497559    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:11:51.165142 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:51.165110    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:51.812649 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:51.812611    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:58.578501 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:11:58.578469    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:01.568118 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:01.568084    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:01.668955 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:01.668900    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:01.744601 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:01.744570    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:01.812114 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:01.812062    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:02.164041 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:02.164008    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:03.047791 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:03.047752    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:03.676781 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:03.676743    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:04.430086 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:04.430053    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:05.139963 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:05.139933    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:05.344086 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:05.344055    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:06.141320 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:06.141279    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:06.768257 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:06.768179    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:10.211269 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:10.211229    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:23.794185 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:23.794147    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:12:46.272394 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:46.272364    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:46.295655 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:46.295619    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:12:47.149692 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:47.149657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:51.106459 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:51.106424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:52.863315 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:52.863283    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:54.739809 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:54.739767    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:54.788827 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:12:54.788781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:03.864840 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:03.864803    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:13.502638 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:13.502604    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:15.239539 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:15.239476    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:15.340481 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:15.340436    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:15.492163 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:15.492084    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"]
Sep 14 08:13:15.492415 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:15.492390    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl" podUID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerName="istio-proxy" containerID="cri-o://89e0229844d78a8e726bea6f9142009e12ee8995b8925a902ad1fb77a6f0b8f1" gracePeriod=30
Sep 14 08:13:15.513417 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:15.513376    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:16.196451 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:16.196421    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:16.831137 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:16.831099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:16.928811 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:16.928776    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:17.028448 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:17.028419    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:17.663667 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:17.663622    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:17.757342 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:17.757306    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:17.854778 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:17.854737    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:18.606660 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:18.606627    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:19.147430 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:19.147399    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:20.739320 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.739297    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:13:20.844625 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.844559    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-token\") pod \"e10e6ba3-55bf-46de-8254-fa798815b1b2\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") "
Sep 14 08:13:20.844625 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.844618    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9fsw\" (UniqueName: \"kubernetes.io/projected/e10e6ba3-55bf-46de-8254-fa798815b1b2-kube-api-access-q9fsw\") pod \"e10e6ba3-55bf-46de-8254-fa798815b1b2\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") "
Sep 14 08:13:20.844811 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.844648    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-podinfo\") pod \"e10e6ba3-55bf-46de-8254-fa798815b1b2\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") "
Sep 14 08:13:20.844874 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.844814    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-workload-certs\") pod \"e10e6ba3-55bf-46de-8254-fa798815b1b2\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") "
Sep 14 08:13:20.844874 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.844850    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-credential-socket\") pod \"e10e6ba3-55bf-46de-8254-fa798815b1b2\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") "
Sep 14 08:13:20.844984 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.844941    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-data\") pod \"e10e6ba3-55bf-46de-8254-fa798815b1b2\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") "
Sep 14 08:13:20.845039 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.844995    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-workload-socket\") pod \"e10e6ba3-55bf-46de-8254-fa798815b1b2\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") "
Sep 14 08:13:20.845039 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.845016    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e10e6ba3-55bf-46de-8254-fa798815b1b2-istiod-ca-cert\") pod \"e10e6ba3-55bf-46de-8254-fa798815b1b2\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") "
Sep 14 08:13:20.845147 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.845044    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-envoy\") pod \"e10e6ba3-55bf-46de-8254-fa798815b1b2\" (UID: \"e10e6ba3-55bf-46de-8254-fa798815b1b2\") "
Sep 14 08:13:20.845147 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.845080    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e10e6ba3-55bf-46de-8254-fa798815b1b2" (UID: "e10e6ba3-55bf-46de-8254-fa798815b1b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:20.845147 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.845119    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e10e6ba3-55bf-46de-8254-fa798815b1b2" (UID: "e10e6ba3-55bf-46de-8254-fa798815b1b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:20.845342 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.845278    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "e10e6ba3-55bf-46de-8254-fa798815b1b2" (UID: "e10e6ba3-55bf-46de-8254-fa798815b1b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:20.845404 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.845319    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e10e6ba3-55bf-46de-8254-fa798815b1b2" (UID: "e10e6ba3-55bf-46de-8254-fa798815b1b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:20.845526 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.845463    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e10e6ba3-55bf-46de-8254-fa798815b1b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e10e6ba3-55bf-46de-8254-fa798815b1b2" (UID: "e10e6ba3-55bf-46de-8254-fa798815b1b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:20.845526 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.845476    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:13:20.845526 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.845494    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:13:20.845526 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.845505    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:13:20.845526 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.845513    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:13:20.847028 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.846998    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e10e6ba3-55bf-46de-8254-fa798815b1b2-kube-api-access-q9fsw" (OuterVolumeSpecName: "kube-api-access-q9fsw") pod "e10e6ba3-55bf-46de-8254-fa798815b1b2" (UID: "e10e6ba3-55bf-46de-8254-fa798815b1b2"). InnerVolumeSpecName "kube-api-access-q9fsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:20.847140 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.847095    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e10e6ba3-55bf-46de-8254-fa798815b1b2" (UID: "e10e6ba3-55bf-46de-8254-fa798815b1b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:20.847337 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.847309    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e10e6ba3-55bf-46de-8254-fa798815b1b2" (UID: "e10e6ba3-55bf-46de-8254-fa798815b1b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:13:20.847438 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.847333    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "e10e6ba3-55bf-46de-8254-fa798815b1b2" (UID: "e10e6ba3-55bf-46de-8254-fa798815b1b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:20.946050 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.946028    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e10e6ba3-55bf-46de-8254-fa798815b1b2-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:13:20.946050 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.946048    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:13:20.946182 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.946057    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:13:20.946182 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.946065    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9fsw\" (UniqueName: \"kubernetes.io/projected/e10e6ba3-55bf-46de-8254-fa798815b1b2-kube-api-access-q9fsw\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:13:20.946182 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:20.946073    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e10e6ba3-55bf-46de-8254-fa798815b1b2-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:13:21.304924 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:21.304886    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:13:21.358994 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:21.358960    2576 generic.go:356] "Generic (PLEG): container finished" podID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerID="89e0229844d78a8e726bea6f9142009e12ee8995b8925a902ad1fb77a6f0b8f1" exitCode=0
Sep 14 08:13:21.359166 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:21.359027    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"
Sep 14 08:13:21.359166 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:21.359050    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl" event={"ID":"e10e6ba3-55bf-46de-8254-fa798815b1b2","Type":"ContainerDied","Data":"89e0229844d78a8e726bea6f9142009e12ee8995b8925a902ad1fb77a6f0b8f1"}
Sep 14 08:13:21.359166 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:21.359107    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl" event={"ID":"e10e6ba3-55bf-46de-8254-fa798815b1b2","Type":"ContainerDied","Data":"9c7b23196843c6054e22af037f689530bffd0e0a37f8e2d40f63e6a7449d164f"}
Sep 14 08:13:21.359166 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:21.359130    2576 scope.go:117] "RemoveContainer" containerID="89e0229844d78a8e726bea6f9142009e12ee8995b8925a902ad1fb77a6f0b8f1"
Sep 14 08:13:21.369069 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:21.369052    2576 scope.go:117] "RemoveContainer" containerID="5c718dcd8b142576709b698a0113c372da2c18b572ffee9c7a4f1a6eebe6f39f"
Sep 14 08:13:21.377855 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:21.377654    2576 scope.go:117] "RemoveContainer" containerID="89e0229844d78a8e726bea6f9142009e12ee8995b8925a902ad1fb77a6f0b8f1"
Sep 14 08:13:21.378068 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:13:21.378024    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89e0229844d78a8e726bea6f9142009e12ee8995b8925a902ad1fb77a6f0b8f1\": container with ID starting with 89e0229844d78a8e726bea6f9142009e12ee8995b8925a902ad1fb77a6f0b8f1 not found: ID does not exist" containerID="89e0229844d78a8e726bea6f9142009e12ee8995b8925a902ad1fb77a6f0b8f1"
Sep 14 08:13:21.378151 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:21.378080    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e0229844d78a8e726bea6f9142009e12ee8995b8925a902ad1fb77a6f0b8f1"} err="failed to get container status \"89e0229844d78a8e726bea6f9142009e12ee8995b8925a902ad1fb77a6f0b8f1\": rpc error: code = NotFound desc = could not find container \"89e0229844d78a8e726bea6f9142009e12ee8995b8925a902ad1fb77a6f0b8f1\": container with ID starting with 89e0229844d78a8e726bea6f9142009e12ee8995b8925a902ad1fb77a6f0b8f1 not found: ID does not exist"
Sep 14 08:13:21.378151 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:21.378106    2576 scope.go:117] "RemoveContainer" containerID="5c718dcd8b142576709b698a0113c372da2c18b572ffee9c7a4f1a6eebe6f39f"
Sep 14 08:13:21.378422 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:13:21.378407    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c718dcd8b142576709b698a0113c372da2c18b572ffee9c7a4f1a6eebe6f39f\": container with ID starting with 5c718dcd8b142576709b698a0113c372da2c18b572ffee9c7a4f1a6eebe6f39f not found: ID does not exist" containerID="5c718dcd8b142576709b698a0113c372da2c18b572ffee9c7a4f1a6eebe6f39f"
Sep 14 08:13:21.378463 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:21.378428    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c718dcd8b142576709b698a0113c372da2c18b572ffee9c7a4f1a6eebe6f39f"} err="failed to get container status \"5c718dcd8b142576709b698a0113c372da2c18b572ffee9c7a4f1a6eebe6f39f\": rpc error: code = NotFound desc = could not find container \"5c718dcd8b142576709b698a0113c372da2c18b572ffee9c7a4f1a6eebe6f39f\": container with ID starting with 5c718dcd8b142576709b698a0113c372da2c18b572ffee9c7a4f1a6eebe6f39f not found: ID does not exist"
Sep 14 08:13:21.380394 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:21.380376    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"]
Sep 14 08:13:21.386014 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:21.385993    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2a94f0ce-openshift-default-64bcc8fcb-rgqcl"]
Sep 14 08:13:21.838958 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:21.838933    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e10e6ba3-55bf-46de-8254-fa798815b1b2" path="/var/lib/kubelet/pods/e10e6ba3-55bf-46de-8254-fa798815b1b2/volumes"
Sep 14 08:13:24.949233 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:24.949178    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:25.051389 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:25.051364    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:25.669475 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:25.669428    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:31.501615 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:31.501579    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:31.590089 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:31.590048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:31.679577 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:31.679548    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:42.178319 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:42.178289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:42.414285 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:42.414238    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:45.020971 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:45.020926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:55.494010 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:13:55.493974    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:14:04.512575 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:04.512538    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:05.957405 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:05.957374    2576 scope.go:117] "RemoveContainer" containerID="b385f3f9ccec3a0aa24eb6af5738bb488c45e73b765c2532943e59c76a2fc78d"
Sep 14 08:14:09.029702 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.029665    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:09.101797 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.101765    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:09.395739 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.395710    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:09.734808 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.734716    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"]
Sep 14 08:14:09.735869 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.735828    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerName="istio-proxy"
Sep 14 08:14:09.735869 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.735861    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerName="istio-proxy"
Sep 14 08:14:09.736083 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.735988    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerName="istio-proxy"
Sep 14 08:14:09.736083 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.736002    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerName="istio-proxy"
Sep 14 08:14:09.739672 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.739651    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:09.749890 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.749866    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-b22414-openshift-default-dockercfg-rkhpz\""
Sep 14 08:14:09.752380 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.752352    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"]
Sep 14 08:14:09.759791 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.759761    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:09.853265 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.853228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:09.928837 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.928799    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-token\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:09.929100 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.929077    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-podinfo\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:09.929257 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.929239    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-envoy\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:09.929392 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.929374    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqs4x\" (UniqueName: \"kubernetes.io/projected/4b043a3b-f9f4-4084-832a-729fdaa16278-kube-api-access-jqs4x\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:09.929523 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.929508    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-data\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:09.929686 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.929671    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-workload-socket\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:09.929794 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.929780    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-credential-socket\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:09.930290 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.930268    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b043a3b-f9f4-4084-832a-729fdaa16278-istiod-ca-cert\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:09.930500 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:09.930485    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-workload-certs\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.031508 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.031410    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-token\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.032065 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.032034    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-podinfo\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.032196 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.032182    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-envoy\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.032339 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.032324    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqs4x\" (UniqueName: \"kubernetes.io/projected/4b043a3b-f9f4-4084-832a-729fdaa16278-kube-api-access-jqs4x\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.032458 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.032445    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-data\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.032615 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.032600    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-workload-socket\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.032728 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.032709    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-credential-socket\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.032890 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.032875    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b043a3b-f9f4-4084-832a-729fdaa16278-istiod-ca-cert\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.033028 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.033015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-workload-certs\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.033182 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.033152    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-data\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.033559 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.033466    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-credential-socket\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.034496 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.034434    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b043a3b-f9f4-4084-832a-729fdaa16278-istiod-ca-cert\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.034901 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.034862    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-workload-socket\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.035278 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.035258    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-workload-certs\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.035747 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.035724    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-envoy\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.035931 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.035881    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-podinfo\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.041393 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.041370    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqs4x\" (UniqueName: \"kubernetes.io/projected/4b043a3b-f9f4-4084-832a-729fdaa16278-kube-api-access-jqs4x\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.041546 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.041506    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-token\") pod \"e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") " pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.067807 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.067516    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:10.390354 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.390329    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"]
Sep 14 08:14:10.392129 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:14:10.392103    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b043a3b_f9f4_4084_832a_729fdaa16278.slice/crio-9becfa8c84cec85bac7604b88eaef2c7dc17c9a7f63fca4cf9ed4cfe7813f99c WatchSource:0}: Error finding container 9becfa8c84cec85bac7604b88eaef2c7dc17c9a7f63fca4cf9ed4cfe7813f99c: Status 404 returned error can't find the container with id 9becfa8c84cec85bac7604b88eaef2c7dc17c9a7f63fca4cf9ed4cfe7813f99c
Sep 14 08:14:10.394480 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.394448    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:14:10.394558 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.394526    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:14:10.394558 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.394553    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:14:10.628241 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.628189    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t" event={"ID":"4b043a3b-f9f4-4084-832a-729fdaa16278","Type":"ContainerStarted","Data":"93391552eeba8d4e94b2a8974479de6b32e733c76d8200c21999fabf56fde8f0"}
Sep 14 08:14:10.628405 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.628260    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t" event={"ID":"4b043a3b-f9f4-4084-832a-729fdaa16278","Type":"ContainerStarted","Data":"9becfa8c84cec85bac7604b88eaef2c7dc17c9a7f63fca4cf9ed4cfe7813f99c"}
Sep 14 08:14:10.645864 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:10.645755    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t" podStartSLOduration=1.645736675 podStartE2EDuration="1.645736675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:14:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:14:10.644898906 +0000 UTC m=+1505.349437410" watchObservedRunningTime="2026-09-14 08:14:10.645736675 +0000 UTC m=+1505.350275178"
Sep 14 08:14:11.068002 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:11.067917    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:11.073249 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:11.073225    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:11.633860 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:11.633826    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:11.634823 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:11.634804    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:11.943061 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:11.942982    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:12.647920 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:12.647866    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:12.745868 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:12.745838    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:12.847935 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:12.847903    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:13.640692 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:13.640659    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:14.388702 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:14.388669    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:15.125086 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:15.125050    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:15.476671 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:15.476592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:16.042056 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:16.042022    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:16.728464 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:16.728431    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:17.350199 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.350164    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"]
Sep 14 08:14:17.350785 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.350757    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerName="istio-proxy"
Sep 14 08:14:17.350785 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.350785    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e10e6ba3-55bf-46de-8254-fa798815b1b2" containerName="istio-proxy"
Sep 14 08:14:17.354432 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.354407    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:17.356849 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.356825    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-b22414-serving-cert\""
Sep 14 08:14:17.362283 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.362199    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"]
Sep 14 08:14:17.386275 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.386233    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:17.411517 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.411484    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-metrics-certs\") pod \"maas-api-e2e-api-a-b22414-59bfdcf9dc-596br\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:17.411685 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.411540    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dswh\" (UniqueName: \"kubernetes.io/projected/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-kube-api-access-8dswh\") pod \"maas-api-e2e-api-a-b22414-59bfdcf9dc-596br\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:17.411685 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.411601    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-openshift-service-ca\") pod \"maas-api-e2e-api-a-b22414-59bfdcf9dc-596br\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:17.411762 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.411694    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-maas-api-tls\") pod \"maas-api-e2e-api-a-b22414-59bfdcf9dc-596br\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:17.445601 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.445567    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:17.512788 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.512747    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-openshift-service-ca\") pod \"maas-api-e2e-api-a-b22414-59bfdcf9dc-596br\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:17.512992 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.512833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-maas-api-tls\") pod \"maas-api-e2e-api-a-b22414-59bfdcf9dc-596br\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:17.512992 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.512924    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-metrics-certs\") pod \"maas-api-e2e-api-a-b22414-59bfdcf9dc-596br\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:17.512992 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.512985    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dswh\" (UniqueName: \"kubernetes.io/projected/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-kube-api-access-8dswh\") pod \"maas-api-e2e-api-a-b22414-59bfdcf9dc-596br\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:17.513137 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:14:17.512987    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-serving-cert: secret "maas-api-e2e-api-a-b22414-serving-cert" not found
Sep 14 08:14:17.513174 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:14:17.513145    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-maas-api-tls podName:1c4d8523-73c9-4e9a-9a9a-2cff902534ca nodeName:}" failed. No retries permitted until 2026-09-14 08:14:18.013119215 +0000 UTC m=+1512.717657717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-maas-api-tls") pod "maas-api-e2e-api-a-b22414-59bfdcf9dc-596br" (UID: "1c4d8523-73c9-4e9a-9a9a-2cff902534ca") : secret "maas-api-e2e-api-a-b22414-serving-cert" not found
Sep 14 08:14:17.513467 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.513448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-openshift-service-ca\") pod \"maas-api-e2e-api-a-b22414-59bfdcf9dc-596br\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:17.515491 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.515470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-metrics-certs\") pod \"maas-api-e2e-api-a-b22414-59bfdcf9dc-596br\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:17.522877 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.522852    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dswh\" (UniqueName: \"kubernetes.io/projected/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-kube-api-access-8dswh\") pod \"maas-api-e2e-api-a-b22414-59bfdcf9dc-596br\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:17.745744 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.745711    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:17.843697 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.843664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:17.935704 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:17.935673    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:18.019254 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:18.019156    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-maas-api-tls\") pod \"maas-api-e2e-api-a-b22414-59bfdcf9dc-596br\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:18.021608 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:18.021580    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-maas-api-tls\") pod \"maas-api-e2e-api-a-b22414-59bfdcf9dc-596br\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:18.268476 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:18.268445    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:18.357928 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:18.357892    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:18.545121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:18.545093    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"]
Sep 14 08:14:18.546418 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:14:18.546389    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c4d8523_73c9_4e9a_9a9a_2cff902534ca.slice/crio-50258995efdb74bcff06fdee117878d4eac1becc65780b2f71916dfa01d2703a WatchSource:0}: Error finding container 50258995efdb74bcff06fdee117878d4eac1becc65780b2f71916dfa01d2703a: Status 404 returned error can't find the container with id 50258995efdb74bcff06fdee117878d4eac1becc65780b2f71916dfa01d2703a
Sep 14 08:14:18.547851 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:18.547834    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:14:18.678061 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:18.678028    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br" event={"ID":"1c4d8523-73c9-4e9a-9a9a-2cff902534ca","Type":"ContainerStarted","Data":"50258995efdb74bcff06fdee117878d4eac1becc65780b2f71916dfa01d2703a"}
Sep 14 08:14:19.048293 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:19.048253    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:19.684682 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:19.684644    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br" event={"ID":"1c4d8523-73c9-4e9a-9a9a-2cff902534ca","Type":"ContainerStarted","Data":"8b57e2812695919e947f6748c74bcbb536293a2a1b5062c088019c16f1b2a24b"}
Sep 14 08:14:19.684835 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:19.684777    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:19.702062 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:19.701996    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br" podStartSLOduration=2.225106339 podStartE2EDuration="2.701980749s" totalImagesPullingTime="476.87441ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:14:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:14:19.699417683 +0000 UTC m=+1514.403956188" watchObservedRunningTime="2026-09-14 08:14:19.701980749 +0000 UTC m=+1514.406519250"
Sep 14 08:14:19.814633 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:19.814588    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:20.559014 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:20.558982    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:20.591574 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:20.591536    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"]
Sep 14 08:14:20.652882 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:20.652854    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:21.238995 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:21.238964    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:21.937685 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:21.937653    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:22.394395 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:22.394367    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:14:22.561316 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:22.561275    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:23.174007 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:23.173961    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:25.695406 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:25.695371    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:14:25.841507 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:25.841475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:31.390415 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.390221    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"]
Sep 14 08:14:31.391435 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.391140    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6" podUID="50945a1e-fa44-49c7-b141-7e29449f1962" containerName="maas-api" containerID="cri-o://6ebc0578fe27ee3f37276cea126d35017ba2b15a994ac8b99e15fde7e7516f13" gracePeriod=30
Sep 14 08:14:31.423865 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.423831    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:31.525113 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.525082    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:31.605735 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.605697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:31.766588 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.766551    2576 generic.go:356] "Generic (PLEG): container finished" podID="50945a1e-fa44-49c7-b141-7e29449f1962" containerID="6ebc0578fe27ee3f37276cea126d35017ba2b15a994ac8b99e15fde7e7516f13" exitCode=0
Sep 14 08:14:31.766774 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.766639    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6" event={"ID":"50945a1e-fa44-49c7-b141-7e29449f1962","Type":"ContainerDied","Data":"6ebc0578fe27ee3f37276cea126d35017ba2b15a994ac8b99e15fde7e7516f13"}
Sep 14 08:14:31.790798 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.790702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:31.867355 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.867331    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:14:31.964863 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.964766    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50945a1e-fa44-49c7-b141-7e29449f1962-openshift-service-ca\") pod \"50945a1e-fa44-49c7-b141-7e29449f1962\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") "
Sep 14 08:14:31.964863 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.964816    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-metrics-certs\") pod \"50945a1e-fa44-49c7-b141-7e29449f1962\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") "
Sep 14 08:14:31.965099 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.964873    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rklqd\" (UniqueName: \"kubernetes.io/projected/50945a1e-fa44-49c7-b141-7e29449f1962-kube-api-access-rklqd\") pod \"50945a1e-fa44-49c7-b141-7e29449f1962\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") "
Sep 14 08:14:31.965099 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.965075    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-maas-api-tls\") pod \"50945a1e-fa44-49c7-b141-7e29449f1962\" (UID: \"50945a1e-fa44-49c7-b141-7e29449f1962\") "
Sep 14 08:14:31.965259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.965232    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50945a1e-fa44-49c7-b141-7e29449f1962-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "50945a1e-fa44-49c7-b141-7e29449f1962" (UID: "50945a1e-fa44-49c7-b141-7e29449f1962"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:14:31.965478 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.965462    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50945a1e-fa44-49c7-b141-7e29449f1962-openshift-service-ca\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:31.967258 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.967200    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "50945a1e-fa44-49c7-b141-7e29449f1962" (UID: "50945a1e-fa44-49c7-b141-7e29449f1962"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:14:31.967548 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.967527    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "50945a1e-fa44-49c7-b141-7e29449f1962" (UID: "50945a1e-fa44-49c7-b141-7e29449f1962"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:14:31.967619 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:31.967552    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50945a1e-fa44-49c7-b141-7e29449f1962-kube-api-access-rklqd" (OuterVolumeSpecName: "kube-api-access-rklqd") pod "50945a1e-fa44-49c7-b141-7e29449f1962" (UID: "50945a1e-fa44-49c7-b141-7e29449f1962"). InnerVolumeSpecName "kube-api-access-rklqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:14:32.001290 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:32.001253    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:32.066832 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:32.066795    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-maas-api-tls\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:32.066832 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:32.066825    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50945a1e-fa44-49c7-b141-7e29449f1962-metrics-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:32.066832 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:32.066834    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rklqd\" (UniqueName: \"kubernetes.io/projected/50945a1e-fa44-49c7-b141-7e29449f1962-kube-api-access-rklqd\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:32.108989 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:32.108954    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:32.744489 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:32.744434    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:32.773428 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:32.773394    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6" event={"ID":"50945a1e-fa44-49c7-b141-7e29449f1962","Type":"ContainerDied","Data":"75fe6136e75f89bb71e3a9bec3190710e24858a9543a9455cc382eb8285f905e"}
Sep 14 08:14:32.773634 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:32.773444    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"
Sep 14 08:14:32.773634 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:32.773506    2576 scope.go:117] "RemoveContainer" containerID="6ebc0578fe27ee3f37276cea126d35017ba2b15a994ac8b99e15fde7e7516f13"
Sep 14 08:14:32.798061 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:32.798034    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"]
Sep 14 08:14:32.800479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:32.800457    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-831f8e-cc966c997-szks6"]
Sep 14 08:14:33.375854 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:33.375802    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:33.839744 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:33.839653    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50945a1e-fa44-49c7-b141-7e29449f1962" path="/var/lib/kubelet/pods/50945a1e-fa44-49c7-b141-7e29449f1962/volumes"
Sep 14 08:14:34.216978 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:34.216941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:34.954905 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:34.954875    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:35.611161 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:35.611101    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7" podUID="26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d" containerName="main" containerID="cri-o://6fe0893c707ae069357a2f13c440375af4ace228eabf182e75ce8643c940a893" gracePeriod=45
Sep 14 08:14:35.636356 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:35.636332    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:35.794879 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:35.794847    2576 generic.go:356] "Generic (PLEG): container finished" podID="26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d" containerID="6fe0893c707ae069357a2f13c440375af4ace228eabf182e75ce8643c940a893" exitCode=0
Sep 14 08:14:35.795030 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:35.794913    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7" event={"ID":"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d","Type":"ContainerDied","Data":"6fe0893c707ae069357a2f13c440375af4ace228eabf182e75ce8643c940a893"}
Sep 14 08:14:35.873376 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:35.873315    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:14:36.005511 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.005471    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-home\") pod \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") "
Sep 14 08:14:36.005925 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.005525    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-tmp-dir\") pod \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") "
Sep 14 08:14:36.005925 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.005595    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-model-cache\") pod \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") "
Sep 14 08:14:36.005925 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.005685    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm99g\" (UniqueName: \"kubernetes.io/projected/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-kube-api-access-wm99g\") pod \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") "
Sep 14 08:14:36.005925 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.005702    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-home" (OuterVolumeSpecName: "home") pod "26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d" (UID: "26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:36.005925 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.005728    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-tls-certs\") pod \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") "
Sep 14 08:14:36.005925 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.005754    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-dshm\") pod \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\" (UID: \"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d\") "
Sep 14 08:14:36.005925 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.005779    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d" (UID: "26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:36.005925 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.005830    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-model-cache" (OuterVolumeSpecName: "model-cache") pod "26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d" (UID: "26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:36.006350 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.006178    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-home\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.006350 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.006200    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-tmp-dir\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.006350 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.006247    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-model-cache\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.007852 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.007830    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d" (UID: "26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:14:36.008089 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.008075    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-dshm" (OuterVolumeSpecName: "dshm") pod "26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d" (UID: "26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:36.008157 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.008138    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-kube-api-access-wm99g" (OuterVolumeSpecName: "kube-api-access-wm99g") pod "26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d" (UID: "26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d"). InnerVolumeSpecName "kube-api-access-wm99g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:14:36.107761 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.107718    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm99g\" (UniqueName: \"kubernetes.io/projected/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-kube-api-access-wm99g\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.107761 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.107754    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-tls-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.107761 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.107764    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d-dshm\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.802714 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.802681    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"
Sep 14 08:14:36.802917 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.802719    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7" event={"ID":"26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d","Type":"ContainerDied","Data":"12adb80c3bbad2db41495d4528e88f6bf356dd6c215e2f8c70e25e234a867f53"}
Sep 14 08:14:36.802917 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.802775    2576 scope.go:117] "RemoveContainer" containerID="6fe0893c707ae069357a2f13c440375af4ace228eabf182e75ce8643c940a893"
Sep 14 08:14:36.826316 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.826272    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"]
Sep 14 08:14:36.828737 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.828714    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/simulator-w0-831f8e-kserve-6d76875986-sq6z7"]
Sep 14 08:14:36.870875 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:36.870840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:37.135444 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:37.135409    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:37.637929 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:37.637894    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:37.838746 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:37.838709    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d" path="/var/lib/kubelet/pods/26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d/volumes"
Sep 14 08:14:37.927608 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:37.927526    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:38.133317 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.133285    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:38.424959 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.424923    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"]
Sep 14 08:14:38.425614 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.425585    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d" containerName="main"
Sep 14 08:14:38.425614 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.425610    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d" containerName="main"
Sep 14 08:14:38.425781 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.425636    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50945a1e-fa44-49c7-b141-7e29449f1962" containerName="maas-api"
Sep 14 08:14:38.425781 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.425645    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="50945a1e-fa44-49c7-b141-7e29449f1962" containerName="maas-api"
Sep 14 08:14:38.425781 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.425737    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="26ff0142-c0d1-4d7e-80bc-7fc494fb7d0d" containerName="main"
Sep 14 08:14:38.425781 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.425750    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="50945a1e-fa44-49c7-b141-7e29449f1962" containerName="maas-api"
Sep 14 08:14:38.429259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.429239    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.432128 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.432106    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-8e721c-openshift-default-dockercfg-q5g9f\""
Sep 14 08:14:38.441492 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.441472    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"]
Sep 14 08:14:38.462017 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.461994    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:38.535400 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.535362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8zgn\" (UniqueName: \"kubernetes.io/projected/0986547a-8cd0-4d9f-9745-bd65f8ad5085-kube-api-access-n8zgn\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.535529 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.535417    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istiod-ca-cert\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.535529 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.535449    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-workload-socket\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.535529 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.535516    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-credential-socket\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.535648 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.535532    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-podinfo\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.535648 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.535568    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-envoy\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.535648 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.535589    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-workload-certs\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.535758 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.535665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-data\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.535758 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.535703    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-token\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.636987 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.636947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-envoy\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.636987 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.636988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-workload-certs\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.637259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.637017    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-data\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.637259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.637039    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-token\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.637259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.637087    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8zgn\" (UniqueName: \"kubernetes.io/projected/0986547a-8cd0-4d9f-9745-bd65f8ad5085-kube-api-access-n8zgn\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.637259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.637133    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istiod-ca-cert\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.637259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.637187    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-workload-socket\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.637553 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.637494    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-workload-certs\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.637553 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.637516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-credential-socket\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.637553 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.637549    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-podinfo\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.637725 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.637660    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-workload-socket\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.637871 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.637850    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-credential-socket\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.637935 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.637902    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istiod-ca-cert\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.638022 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.638006    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-data\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.639313 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.639294    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-envoy\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.640042 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.640022    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-podinfo\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.645908 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.645887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-token\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.646294 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.646272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8zgn\" (UniqueName: \"kubernetes.io/projected/0986547a-8cd0-4d9f-9745-bd65f8ad5085-kube-api-access-n8zgn\") pod \"e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") " pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.662979 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.662939    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:38.740762 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.740681    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:38.758237 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.758186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:38.942429 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.942397    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"]
Sep 14 08:14:38.942612 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.942590    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerName="istio-proxy" containerID="cri-o://68cc4f943aa36c5748eb50a291f7db53139b5772b7698396aca84fa07b825c2d" gracePeriod=30
Sep 14 08:14:38.977509 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:38.977478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:39.049711 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:39.049623    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"]
Sep 14 08:14:39.056045 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:14:39.056018    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0986547a_8cd0_4d9f_9745_bd65f8ad5085.slice/crio-b3d33544a6126de31c5d17bc763a5df5a35d33806ac3758ca6b3e2fe4c72b381 WatchSource:0}: Error finding container b3d33544a6126de31c5d17bc763a5df5a35d33806ac3758ca6b3e2fe4c72b381: Status 404 returned error can't find the container with id b3d33544a6126de31c5d17bc763a5df5a35d33806ac3758ca6b3e2fe4c72b381
Sep 14 08:14:39.058571 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:39.058540    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:14:39.058659 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:39.058604    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:14:39.058659 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:39.058633    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:14:39.699728 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:39.699695    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:39.820732 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:39.820689    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" event={"ID":"0986547a-8cd0-4d9f-9745-bd65f8ad5085","Type":"ContainerStarted","Data":"6334361d545da6845fe87b97dfa5a3ca3bc24b920e96614bbdcfa035da77c8c9"}
Sep 14 08:14:39.820895 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:39.820760    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" event={"ID":"0986547a-8cd0-4d9f-9745-bd65f8ad5085","Type":"ContainerStarted","Data":"b3d33544a6126de31c5d17bc763a5df5a35d33806ac3758ca6b3e2fe4c72b381"}
Sep 14 08:14:39.840034 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:39.839978    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" podStartSLOduration=1.839962729 podStartE2EDuration="1.839962729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:14:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:14:39.83794332 +0000 UTC m=+1534.542481858" watchObservedRunningTime="2026-09-14 08:14:39.839962729 +0000 UTC m=+1534.544501232"
Sep 14 08:14:40.445746 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:40.445703    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:40.741559 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:40.741469    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:40.748969 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:40.748921    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:40.827569 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:40.827537    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:40.828618 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:40.828593    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:14:41.385860 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:41.385828    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:41.864000 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:41.863952    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:41.964435 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:41.964407    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:42.346662 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:42.346560    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:43.028739 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:43.028701    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:43.852045 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:43.851998    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:44.211773 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.211745    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:14:44.299563 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.299531    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-credential-socket\") pod \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") "
Sep 14 08:14:44.299720 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.299573    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-podinfo\") pod \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") "
Sep 14 08:14:44.299720 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.299603    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-data\") pod \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") "
Sep 14 08:14:44.299720 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.299637    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istiod-ca-cert\") pod \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") "
Sep 14 08:14:44.299720 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.299705    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-workload-socket\") pod \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") "
Sep 14 08:14:44.299937 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.299740    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-token\") pod \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") "
Sep 14 08:14:44.299937 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.299766    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-workload-certs\") pod \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") "
Sep 14 08:14:44.299937 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.299796    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" (UID: "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:44.299937 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.299867    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-data" (OuterVolumeSpecName: "istio-data") pod "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" (UID: "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:44.300144 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.299998    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d86n7\" (UniqueName: \"kubernetes.io/projected/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-kube-api-access-d86n7\") pod \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") "
Sep 14 08:14:44.300144 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.300036    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-envoy\") pod \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\" (UID: \"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d\") "
Sep 14 08:14:44.300144 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.300049    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" (UID: "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:44.300144 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.300082    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" (UID: "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:44.300144 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.300092    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" (UID: "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:14:44.300480 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.300461    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:44.300540 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.300485    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:44.300540 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.300500    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:44.300540 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.300512    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:44.300540 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.300525    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:44.301904 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.301877    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" (UID: "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:14:44.302023 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.301974    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-token" (OuterVolumeSpecName: "istio-token") pod "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" (UID: "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:14:44.302023 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.302008    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-kube-api-access-d86n7" (OuterVolumeSpecName: "kube-api-access-d86n7") pod "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" (UID: "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d"). InnerVolumeSpecName "kube-api-access-d86n7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:14:44.302169 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.302153    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" (UID: "553aeccf-0f9f-4cde-9b5c-3b6bd897da0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:44.401328 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.401237    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:44.401328 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.401272    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:44.401328 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.401282    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d86n7\" (UniqueName: \"kubernetes.io/projected/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-kube-api-access-d86n7\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:44.401328 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.401291    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:14:44.534635 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.534424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:44.863157 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.863127    2576 generic.go:356] "Generic (PLEG): container finished" podID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerID="68cc4f943aa36c5748eb50a291f7db53139b5772b7698396aca84fa07b825c2d" exitCode=0
Sep 14 08:14:44.863385 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.863183    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"
Sep 14 08:14:44.863385 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.863224    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" event={"ID":"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d","Type":"ContainerDied","Data":"68cc4f943aa36c5748eb50a291f7db53139b5772b7698396aca84fa07b825c2d"}
Sep 14 08:14:44.863385 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.863277    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx" event={"ID":"553aeccf-0f9f-4cde-9b5c-3b6bd897da0d","Type":"ContainerDied","Data":"c81124c6de28febbe5350c94736931d017c4ea3b1f78b48bf16dc13efbccade9"}
Sep 14 08:14:44.863385 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.863300    2576 scope.go:117] "RemoveContainer" containerID="68cc4f943aa36c5748eb50a291f7db53139b5772b7698396aca84fa07b825c2d"
Sep 14 08:14:44.873057 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.873040    2576 scope.go:117] "RemoveContainer" containerID="f752457f1bb0ffcb31a9f5967320305c45df82d61503edcc503faa48af675dac"
Sep 14 08:14:44.880811 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.880794    2576 scope.go:117] "RemoveContainer" containerID="68cc4f943aa36c5748eb50a291f7db53139b5772b7698396aca84fa07b825c2d"
Sep 14 08:14:44.881055 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:14:44.881036    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68cc4f943aa36c5748eb50a291f7db53139b5772b7698396aca84fa07b825c2d\": container with ID starting with 68cc4f943aa36c5748eb50a291f7db53139b5772b7698396aca84fa07b825c2d not found: ID does not exist" containerID="68cc4f943aa36c5748eb50a291f7db53139b5772b7698396aca84fa07b825c2d"
Sep 14 08:14:44.881096 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.881064    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68cc4f943aa36c5748eb50a291f7db53139b5772b7698396aca84fa07b825c2d"} err="failed to get container status \"68cc4f943aa36c5748eb50a291f7db53139b5772b7698396aca84fa07b825c2d\": rpc error: code = NotFound desc = could not find container \"68cc4f943aa36c5748eb50a291f7db53139b5772b7698396aca84fa07b825c2d\": container with ID starting with 68cc4f943aa36c5748eb50a291f7db53139b5772b7698396aca84fa07b825c2d not found: ID does not exist"
Sep 14 08:14:44.881096 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.881081    2576 scope.go:117] "RemoveContainer" containerID="f752457f1bb0ffcb31a9f5967320305c45df82d61503edcc503faa48af675dac"
Sep 14 08:14:44.881335 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:14:44.881317    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f752457f1bb0ffcb31a9f5967320305c45df82d61503edcc503faa48af675dac\": container with ID starting with f752457f1bb0ffcb31a9f5967320305c45df82d61503edcc503faa48af675dac not found: ID does not exist" containerID="f752457f1bb0ffcb31a9f5967320305c45df82d61503edcc503faa48af675dac"
Sep 14 08:14:44.881384 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.881341    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f752457f1bb0ffcb31a9f5967320305c45df82d61503edcc503faa48af675dac"} err="failed to get container status \"f752457f1bb0ffcb31a9f5967320305c45df82d61503edcc503faa48af675dac\": rpc error: code = NotFound desc = could not find container \"f752457f1bb0ffcb31a9f5967320305c45df82d61503edcc503faa48af675dac\": container with ID starting with f752457f1bb0ffcb31a9f5967320305c45df82d61503edcc503faa48af675dac not found: ID does not exist"
Sep 14 08:14:44.885998 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.885974    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"]
Sep 14 08:14:44.891337 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:44.891314    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-831f8e-openshift-default-854875654b-bcfmx"]
Sep 14 08:14:45.252685 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:45.252603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:45.847630 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:45.847594    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" path="/var/lib/kubelet/pods/553aeccf-0f9f-4cde-9b5c-3b6bd897da0d/volumes"
Sep 14 08:14:46.025262 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:46.025195    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:46.669931 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:46.669889    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:46.771038 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:46.770960    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:46.825307 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:46.825281    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:47.117584 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:47.117553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:47.230463 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:47.230427    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:49.895365 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:49.895289    2576 generic.go:356] "Generic (PLEG): container finished" podID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerID="93391552eeba8d4e94b2a8974479de6b32e733c76d8200c21999fabf56fde8f0" exitCode=0
Sep 14 08:14:49.895365 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:49.895356    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t" event={"ID":"4b043a3b-f9f4-4084-832a-729fdaa16278","Type":"ContainerDied","Data":"93391552eeba8d4e94b2a8974479de6b32e733c76d8200c21999fabf56fde8f0"}
Sep 14 08:14:49.895820 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:49.895800    2576 scope.go:117] "RemoveContainer" containerID="93391552eeba8d4e94b2a8974479de6b32e733c76d8200c21999fabf56fde8f0"
Sep 14 08:14:49.896995 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:49.896967    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:14:49.897060 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:49.897033    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:14:49.897097 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:49.897066    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:14:50.902528 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:50.902497    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t" event={"ID":"4b043a3b-f9f4-4084-832a-729fdaa16278","Type":"ContainerStarted","Data":"6bfaa81ca4a86fa635e1fecb64d22540477f13f6908ef014406415f9f5d3ba31"}
Sep 14 08:14:51.068954 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:51.068910    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:51.069179 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:51.069154    2576 patch_prober.go:28] interesting pod/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:14:51.069295 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:51.069270    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t" podUID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 14 08:14:52.068982 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:52.068944    2576 patch_prober.go:28] interesting pod/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:14:52.069402 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:52.069008    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t" podUID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 14 08:14:52.345677 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:52.345448    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:53.069144 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:53.069103    2576 patch_prober.go:28] interesting pod/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:14:53.069651 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:53.069187    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t" podUID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 14 08:14:54.068581 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:54.068544    2576 patch_prober.go:28] interesting pod/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:14:54.068755 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:54.068598    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t" podUID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 14 08:14:55.068926 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:55.068889    2576 patch_prober.go:28] interesting pod/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:14:55.069444 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:55.068952    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t" podUID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 14 08:14:56.072590 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:56.072555    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:56.073013 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:56.072833    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:56.073571 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:56.073548    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:14:57.866236 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:57.866175    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:58.000502 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:58.000473    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:58.101849 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:58.101822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:59.399005 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:14:59.398969    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:15:00.140329 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.140294    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0b8346-29822895-b7nrg"]
Sep 14 08:15:00.140841 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.140815    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerName="istio-proxy"
Sep 14 08:15:00.140841 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.140839    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerName="istio-proxy"
Sep 14 08:15:00.141187 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.140851    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerName="istio-proxy"
Sep 14 08:15:00.141187 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.140861    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerName="istio-proxy"
Sep 14 08:15:00.141187 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.141014    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerName="istio-proxy"
Sep 14 08:15:00.141187 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.141032    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="553aeccf-0f9f-4cde-9b5c-3b6bd897da0d" containerName="istio-proxy"
Sep 14 08:15:00.144513 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.144493    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd7582-29822895-9kbdx"]
Sep 14 08:15:00.144619 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.144598    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0b8346-29822895-b7nrg"
Sep 14 08:15:00.146966 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.146947    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd7582-29822895-9kbdx"
Sep 14 08:15:00.147066 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.146948    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:15:00.155903 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.155883    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0b8346-29822895-b7nrg"]
Sep 14 08:15:00.158306 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.158287    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd7582-29822895-9kbdx"]
Sep 14 08:15:00.158306 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.158302    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0b8346-29822895-b7nrg"
Sep 14 08:15:00.168389 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.168367    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd7582-29822895-9kbdx"
Sep 14 08:15:00.171127 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.171107    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-b22414-29822895-4jtpf"]
Sep 14 08:15:00.183386 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.183365    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-b22414-29822895-4jtpf"]
Sep 14 08:15:00.183530 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.183383    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-b22414-29822895-4jtpf"
Sep 14 08:15:00.213980 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.213786    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-b22414-29822895-4jtpf"
Sep 14 08:15:00.450656 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.450629    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0b8346-29822895-b7nrg"]
Sep 14 08:15:00.452391 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:15:00.452361    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70ee8f1e_8ea8_4646_91d7_66f8c00a6366.slice/crio-7dbbcb7fe43b1ff4ae006938e796c46eaab00da295b377361435117d98b0959b WatchSource:0}: Error finding container 7dbbcb7fe43b1ff4ae006938e796c46eaab00da295b377361435117d98b0959b: Status 404 returned error can't find the container with id 7dbbcb7fe43b1ff4ae006938e796c46eaab00da295b377361435117d98b0959b
Sep 14 08:15:00.459824 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.459737    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd7582-29822895-9kbdx"]
Sep 14 08:15:00.464560 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:15:00.464537    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd779413a_6ffd_4915_b201_ba4e97923d8f.slice/crio-319319b2080ae8aa9b10f137258f12b60afc950e0c2c01a343b7716641534b7c WatchSource:0}: Error finding container 319319b2080ae8aa9b10f137258f12b60afc950e0c2c01a343b7716641534b7c: Status 404 returned error can't find the container with id 319319b2080ae8aa9b10f137258f12b60afc950e0c2c01a343b7716641534b7c
Sep 14 08:15:00.503307 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.503280    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-b22414-29822895-4jtpf"]
Sep 14 08:15:00.504585 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:15:00.504560    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eea8832_21f1_418f_bd87_c6e4a8758038.slice/crio-0d42fce7e53729952b475bcbdc0f3be26732912f5077ca4a17de95e64a4b09a7 WatchSource:0}: Error finding container 0d42fce7e53729952b475bcbdc0f3be26732912f5077ca4a17de95e64a4b09a7: Status 404 returned error can't find the container with id 0d42fce7e53729952b475bcbdc0f3be26732912f5077ca4a17de95e64a4b09a7
Sep 14 08:15:00.966388 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.966342    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0b8346-29822895-b7nrg" event={"ID":"70ee8f1e-8ea8-4646-91d7-66f8c00a6366","Type":"ContainerStarted","Data":"18a3d1a9933060c0f156362fce4650a79e28083103fbb2b9efa10f9ae3545782"}
Sep 14 08:15:00.966599 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.966408    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0b8346-29822895-b7nrg" event={"ID":"70ee8f1e-8ea8-4646-91d7-66f8c00a6366","Type":"ContainerStarted","Data":"7dbbcb7fe43b1ff4ae006938e796c46eaab00da295b377361435117d98b0959b"}
Sep 14 08:15:00.968153 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.968118    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd7582-29822895-9kbdx" event={"ID":"d779413a-6ffd-4915-b201-ba4e97923d8f","Type":"ContainerStarted","Data":"8ec50cad890ae4888723ad2bb3f52dcc4d65ededa210939c6d673cb0b3601e0d"}
Sep 14 08:15:00.968337 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.968169    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd7582-29822895-9kbdx" event={"ID":"d779413a-6ffd-4915-b201-ba4e97923d8f","Type":"ContainerStarted","Data":"319319b2080ae8aa9b10f137258f12b60afc950e0c2c01a343b7716641534b7c"}
Sep 14 08:15:00.970011 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.969979    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-b22414-29822895-4jtpf" event={"ID":"8eea8832-21f1-418f-bd87-c6e4a8758038","Type":"ContainerStarted","Data":"c2d32ba4f8287445a82645f0ca28b1d56970c3d5fd52d18ce5e09fd11bb15980"}
Sep 14 08:15:00.970130 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.970033    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-b22414-29822895-4jtpf" event={"ID":"8eea8832-21f1-418f-bd87-c6e4a8758038","Type":"ContainerStarted","Data":"0d42fce7e53729952b475bcbdc0f3be26732912f5077ca4a17de95e64a4b09a7"}
Sep 14 08:15:00.982123 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.982030    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0b8346-29822895-b7nrg" podStartSLOduration=0.982015342 podStartE2EDuration="982.015342ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:15:00.980245938 +0000 UTC m=+1555.684784435" watchObservedRunningTime="2026-09-14 08:15:00.982015342 +0000 UTC m=+1555.686553844"
Sep 14 08:15:00.996841 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:00.996772    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd7582-29822895-9kbdx" podStartSLOduration=0.996751936 podStartE2EDuration="996.751936ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:15:00.995567144 +0000 UTC m=+1555.700105659" watchObservedRunningTime="2026-09-14 08:15:00.996751936 +0000 UTC m=+1555.701290438"
Sep 14 08:15:01.013773 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:01.013711    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-b22414-29822895-4jtpf" podStartSLOduration=1.013690711 podStartE2EDuration="1.013690711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:15:01.012790617 +0000 UTC m=+1555.717329125" watchObservedRunningTime="2026-09-14 08:15:01.013690711 +0000 UTC m=+1555.718229213"
Sep 14 08:15:01.977129 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:01.977035    2576 generic.go:356] "Generic (PLEG): container finished" podID="8eea8832-21f1-418f-bd87-c6e4a8758038" containerID="c2d32ba4f8287445a82645f0ca28b1d56970c3d5fd52d18ce5e09fd11bb15980" exitCode=0
Sep 14 08:15:01.977129 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:01.977114    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-b22414-29822895-4jtpf" event={"ID":"8eea8832-21f1-418f-bd87-c6e4a8758038","Type":"ContainerDied","Data":"c2d32ba4f8287445a82645f0ca28b1d56970c3d5fd52d18ce5e09fd11bb15980"}
Sep 14 08:15:01.978924 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:01.978901    2576 generic.go:356] "Generic (PLEG): container finished" podID="70ee8f1e-8ea8-4646-91d7-66f8c00a6366" containerID="18a3d1a9933060c0f156362fce4650a79e28083103fbb2b9efa10f9ae3545782" exitCode=0
Sep 14 08:15:01.979054 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:01.978990    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0b8346-29822895-b7nrg" event={"ID":"70ee8f1e-8ea8-4646-91d7-66f8c00a6366","Type":"ContainerDied","Data":"18a3d1a9933060c0f156362fce4650a79e28083103fbb2b9efa10f9ae3545782"}
Sep 14 08:15:01.980780 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:01.980757    2576 generic.go:356] "Generic (PLEG): container finished" podID="d779413a-6ffd-4915-b201-ba4e97923d8f" containerID="8ec50cad890ae4888723ad2bb3f52dcc4d65ededa210939c6d673cb0b3601e0d" exitCode=0
Sep 14 08:15:01.980882 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:01.980812    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd7582-29822895-9kbdx" event={"ID":"d779413a-6ffd-4915-b201-ba4e97923d8f","Type":"ContainerDied","Data":"8ec50cad890ae4888723ad2bb3f52dcc4d65ededa210939c6d673cb0b3601e0d"}
Sep 14 08:15:02.134841 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:02.134795    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:03.175251 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:03.175227    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0b8346-29822895-b7nrg"
Sep 14 08:15:03.246327 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:03.245379    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-b22414-29822895-4jtpf"
Sep 14 08:15:03.247333 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:03.247258    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd7582-29822895-9kbdx"
Sep 14 08:15:03.993144 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:03.993113    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd7582-29822895-9kbdx"
Sep 14 08:15:03.993144 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:03.993132    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd7582-29822895-9kbdx" event={"ID":"d779413a-6ffd-4915-b201-ba4e97923d8f","Type":"ContainerDied","Data":"319319b2080ae8aa9b10f137258f12b60afc950e0c2c01a343b7716641534b7c"}
Sep 14 08:15:03.993359 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:03.993176    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="319319b2080ae8aa9b10f137258f12b60afc950e0c2c01a343b7716641534b7c"
Sep 14 08:15:03.994471 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:03.994440    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-b22414-29822895-4jtpf"
Sep 14 08:15:03.994614 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:03.994489    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-b22414-29822895-4jtpf" event={"ID":"8eea8832-21f1-418f-bd87-c6e4a8758038","Type":"ContainerDied","Data":"0d42fce7e53729952b475bcbdc0f3be26732912f5077ca4a17de95e64a4b09a7"}
Sep 14 08:15:03.994614 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:03.994533    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d42fce7e53729952b475bcbdc0f3be26732912f5077ca4a17de95e64a4b09a7"
Sep 14 08:15:03.995836 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:03.995816    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0b8346-29822895-b7nrg"
Sep 14 08:15:03.995947 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:03.995817    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0b8346-29822895-b7nrg" event={"ID":"70ee8f1e-8ea8-4646-91d7-66f8c00a6366","Type":"ContainerDied","Data":"7dbbcb7fe43b1ff4ae006938e796c46eaab00da295b377361435117d98b0959b"}
Sep 14 08:15:03.995947 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:03.995945    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7dbbcb7fe43b1ff4ae006938e796c46eaab00da295b377361435117d98b0959b"
Sep 14 08:15:04.059762 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:04.059732    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:08.541581 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:08.541545    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:09.722960 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:09.722912    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:09.824594 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:09.824560    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:10.547217 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:10.547184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:10.829830 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:10.829758    2576 patch_prober.go:28] interesting pod/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 14 08:15:10.829830 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:10.829808    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 14 08:15:11.034712 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:11.034677    2576 generic.go:356] "Generic (PLEG): container finished" podID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerID="6334361d545da6845fe87b97dfa5a3ca3bc24b920e96614bbdcfa035da77c8c9" exitCode=0
Sep 14 08:15:11.034878 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:11.034752    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" event={"ID":"0986547a-8cd0-4d9f-9745-bd65f8ad5085","Type":"ContainerDied","Data":"6334361d545da6845fe87b97dfa5a3ca3bc24b920e96614bbdcfa035da77c8c9"}
Sep 14 08:15:11.035236 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:11.035198    2576 scope.go:117] "RemoveContainer" containerID="6334361d545da6845fe87b97dfa5a3ca3bc24b920e96614bbdcfa035da77c8c9"
Sep 14 08:15:11.036442 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:11.036409    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:15:11.036505 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:11.036485    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:15:11.036558 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:11.036517    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:15:11.245781 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:11.245740    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:11.883938 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:11.883905    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:12.046721 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:12.046678    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" event={"ID":"0986547a-8cd0-4d9f-9745-bd65f8ad5085","Type":"ContainerStarted","Data":"b24277a4106c19b2093f2e553eb0ff84af6c6477fb4747dbedfb44018da39fbf"}
Sep 14 08:15:12.149866 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:12.149783    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:12.741453 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:12.741410    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:15:12.741825 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:12.741794    2576 patch_prober.go:28] interesting pod/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 14 08:15:12.741918 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:12.741853    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 14 08:15:12.919814 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:12.919781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:13.741805 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:13.741769    2576 patch_prober.go:28] interesting pod/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 14 08:15:13.742078 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:13.742049    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 14 08:15:13.766371 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:13.766351    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:14.531199 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:14.531163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:14.741741 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:14.741702    2576 patch_prober.go:28] interesting pod/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 14 08:15:14.742485 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:14.742446    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 14 08:15:15.338033 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:15.337992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:15.742188 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:15.742136    2576 patch_prober.go:28] interesting pod/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 14 08:15:15.742719 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:15.742265    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 14 08:15:16.002267 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:16.002151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:16.635359 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:16.635328    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:16.741950 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:16.741910    2576 patch_prober.go:28] interesting pod/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 14 08:15:16.742268 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:16.742201    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 14 08:15:17.345903 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:17.345878    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:17.745902 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:17.745866    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:15:17.746293 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:17.746250    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:15:17.746928 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:17.746907    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:15:17.751925 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:17.751905    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:17.852383 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:17.852343    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:17.998763 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:17.998671    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:18.636990 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:18.636962    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:19.406830 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:19.406797    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:20.025957 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:20.025926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:20.626808 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:20.626745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:21.250447 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:21.250406    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:15:25.313344 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:25.313313    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:29.828957 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:29.828920    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:30.956087 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:30.956040    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"]
Sep 14 08:15:30.956546 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:30.956324    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br" podUID="1c4d8523-73c9-4e9a-9a9a-2cff902534ca" containerName="maas-api" containerID="cri-o://8b57e2812695919e947f6748c74bcbb536293a2a1b5062c088019c16f1b2a24b" gracePeriod=30
Sep 14 08:15:30.988841 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:30.988799    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:30.989097 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:30.989073    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-b22414-29822895-4jtpf"]
Sep 14 08:15:30.991328 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:30.991304    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-b22414-29822895-4jtpf"]
Sep 14 08:15:31.055217 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.055186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:31.117351 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.117321    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:31.164091 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.164063    2576 generic.go:356] "Generic (PLEG): container finished" podID="1c4d8523-73c9-4e9a-9a9a-2cff902534ca" containerID="8b57e2812695919e947f6748c74bcbb536293a2a1b5062c088019c16f1b2a24b" exitCode=0
Sep 14 08:15:31.164262 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.164143    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br" event={"ID":"1c4d8523-73c9-4e9a-9a9a-2cff902534ca","Type":"ContainerDied","Data":"8b57e2812695919e947f6748c74bcbb536293a2a1b5062c088019c16f1b2a24b"}
Sep 14 08:15:31.181914 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.181887    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:31.212015 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.211962    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:15:31.243283 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.243241    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:31.309322 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.309296    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dswh\" (UniqueName: \"kubernetes.io/projected/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-kube-api-access-8dswh\") pod \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") "
Sep 14 08:15:31.309449 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.309351    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-metrics-certs\") pod \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") "
Sep 14 08:15:31.309449 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.309398    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-maas-api-tls\") pod \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") "
Sep 14 08:15:31.309538 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.309521    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-openshift-service-ca\") pod \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\" (UID: \"1c4d8523-73c9-4e9a-9a9a-2cff902534ca\") "
Sep 14 08:15:31.309923 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.309896    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1c4d8523-73c9-4e9a-9a9a-2cff902534ca" (UID: "1c4d8523-73c9-4e9a-9a9a-2cff902534ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:15:31.311567 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.311541    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-kube-api-access-8dswh" (OuterVolumeSpecName: "kube-api-access-8dswh") pod "1c4d8523-73c9-4e9a-9a9a-2cff902534ca" (UID: "1c4d8523-73c9-4e9a-9a9a-2cff902534ca"). InnerVolumeSpecName "kube-api-access-8dswh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:15:31.311665 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.311572    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1c4d8523-73c9-4e9a-9a9a-2cff902534ca" (UID: "1c4d8523-73c9-4e9a-9a9a-2cff902534ca"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:15:31.311665 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.311598    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1c4d8523-73c9-4e9a-9a9a-2cff902534ca" (UID: "1c4d8523-73c9-4e9a-9a9a-2cff902534ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:15:31.412170 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.412070    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dswh\" (UniqueName: \"kubernetes.io/projected/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-kube-api-access-8dswh\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:15:31.412170 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.412110    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-metrics-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:15:31.412170 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.412127    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-maas-api-tls\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:15:31.412170 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.412139    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c4d8523-73c9-4e9a-9a9a-2cff902534ca-openshift-service-ca\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:15:31.574101 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.573999    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:31.674660 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.674625    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:31.838453 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:31.838378    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eea8832-21f1-418f-bd87-c6e4a8758038" path="/var/lib/kubelet/pods/8eea8832-21f1-418f-bd87-c6e4a8758038/volumes"
Sep 14 08:15:32.170871 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:32.170835    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br" event={"ID":"1c4d8523-73c9-4e9a-9a9a-2cff902534ca","Type":"ContainerDied","Data":"50258995efdb74bcff06fdee117878d4eac1becc65780b2f71916dfa01d2703a"}
Sep 14 08:15:32.171342 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:32.170898    2576 scope.go:117] "RemoveContainer" containerID="8b57e2812695919e947f6748c74bcbb536293a2a1b5062c088019c16f1b2a24b"
Sep 14 08:15:32.171342 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:32.170850    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"
Sep 14 08:15:32.189257 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:32.189227    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"]
Sep 14 08:15:32.192375 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:32.192354    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b22414-59bfdcf9dc-596br"]
Sep 14 08:15:32.352365 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:32.352336    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:33.003547 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:33.003512    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:33.649146 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:33.649112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:33.839494 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:33.839460    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c4d8523-73c9-4e9a-9a9a-2cff902534ca" path="/var/lib/kubelet/pods/1c4d8523-73c9-4e9a-9a9a-2cff902534ca/volumes"
Sep 14 08:15:34.533187 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:34.533151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:35.644454 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:35.644421    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:35.936108 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:35.936023    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:37.658179 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:37.658149    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:37.757353 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:37.757323    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:37.902159 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:37.902123    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"]
Sep 14 08:15:37.902424 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:37.902398    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t" podUID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerName="istio-proxy" containerID="cri-o://6bfaa81ca4a86fa635e1fecb64d22540477f13f6908ef014406415f9f5d3ba31" gracePeriod=30
Sep 14 08:15:37.945130 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:37.945052    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:38.271756 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.271666    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5"]
Sep 14 08:15:38.272085 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.272073    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d779413a-6ffd-4915-b201-ba4e97923d8f" containerName="cleanup"
Sep 14 08:15:38.272129 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.272087    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d779413a-6ffd-4915-b201-ba4e97923d8f" containerName="cleanup"
Sep 14 08:15:38.272129 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.272102    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70ee8f1e-8ea8-4646-91d7-66f8c00a6366" containerName="cleanup"
Sep 14 08:15:38.272129 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.272110    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="70ee8f1e-8ea8-4646-91d7-66f8c00a6366" containerName="cleanup"
Sep 14 08:15:38.272129 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.272124    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c4d8523-73c9-4e9a-9a9a-2cff902534ca" containerName="maas-api"
Sep 14 08:15:38.272129 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.272129    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c4d8523-73c9-4e9a-9a9a-2cff902534ca" containerName="maas-api"
Sep 14 08:15:38.272315 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.272142    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eea8832-21f1-418f-bd87-c6e4a8758038" containerName="cleanup"
Sep 14 08:15:38.272315 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.272148    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eea8832-21f1-418f-bd87-c6e4a8758038" containerName="cleanup"
Sep 14 08:15:38.272315 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.272224    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c4d8523-73c9-4e9a-9a9a-2cff902534ca" containerName="maas-api"
Sep 14 08:15:38.272315 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.272239    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="70ee8f1e-8ea8-4646-91d7-66f8c00a6366" containerName="cleanup"
Sep 14 08:15:38.272315 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.272248    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eea8832-21f1-418f-bd87-c6e4a8758038" containerName="cleanup"
Sep 14 08:15:38.272315 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.272258    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d779413a-6ffd-4915-b201-ba4e97923d8f" containerName="cleanup"
Sep 14 08:15:38.274536 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.274520    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5"
Sep 14 08:15:38.277117 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.277097    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:15:38.282372 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.282343    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5"]
Sep 14 08:15:38.387237 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.387163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6904f040-1cf1-4f5f-a755-1e24125e03dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8e721c-htpj5\" (UID: \"6904f040-1cf1-4f5f-a755-1e24125e03dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5"
Sep 14 08:15:38.488132 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.488092    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6904f040-1cf1-4f5f-a755-1e24125e03dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8e721c-htpj5\" (UID: \"6904f040-1cf1-4f5f-a755-1e24125e03dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5"
Sep 14 08:15:38.488738 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.488717    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6904f040-1cf1-4f5f-a755-1e24125e03dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8e721c-htpj5\" (UID: \"6904f040-1cf1-4f5f-a755-1e24125e03dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5"
Sep 14 08:15:38.536155 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.536066    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:38.587063 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.587029    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5"
Sep 14 08:15:38.866037 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:38.866001    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5"]
Sep 14 08:15:38.867966 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:15:38.867937    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6904f040_1cf1_4f5f_a755_1e24125e03dd.slice/crio-8ea5387a370760ef994bd8d55b30c1292fc0a1d73a68a2fdfeb6b24392160377 WatchSource:0}: Error finding container 8ea5387a370760ef994bd8d55b30c1292fc0a1d73a68a2fdfeb6b24392160377: Status 404 returned error can't find the container with id 8ea5387a370760ef994bd8d55b30c1292fc0a1d73a68a2fdfeb6b24392160377
Sep 14 08:15:39.210960 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:39.210924    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5" event={"ID":"6904f040-1cf1-4f5f-a755-1e24125e03dd","Type":"ContainerStarted","Data":"3c4b12f0ad1bd5dd5378e75f13a9047f160160e9c28dfdbd53bef4eea928abb0"}
Sep 14 08:15:39.211126 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:39.210984    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5" event={"ID":"6904f040-1cf1-4f5f-a755-1e24125e03dd","Type":"ContainerStarted","Data":"8ea5387a370760ef994bd8d55b30c1292fc0a1d73a68a2fdfeb6b24392160377"}
Sep 14 08:15:39.227723 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:39.227668    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5" podStartSLOduration=1.22764758 podStartE2EDuration="1.22764758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:15:39.225151943 +0000 UTC m=+1593.929690484" watchObservedRunningTime="2026-09-14 08:15:39.22764758 +0000 UTC m=+1593.932186081"
Sep 14 08:15:39.348132 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:39.348086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:39.999174 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:39.999142    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:40.217618 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:40.217585    2576 generic.go:356] "Generic (PLEG): container finished" podID="6904f040-1cf1-4f5f-a755-1e24125e03dd" containerID="3c4b12f0ad1bd5dd5378e75f13a9047f160160e9c28dfdbd53bef4eea928abb0" exitCode=0
Sep 14 08:15:40.217791 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:40.217628    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5" event={"ID":"6904f040-1cf1-4f5f-a755-1e24125e03dd","Type":"ContainerDied","Data":"3c4b12f0ad1bd5dd5378e75f13a9047f160160e9c28dfdbd53bef4eea928abb0"}
Sep 14 08:15:40.827977 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:40.827945    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:41.076002 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:41.075968    2576 patch_prober.go:28] interesting pod/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 08:15:41.076384 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:41.076025    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t" podUID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:15:41.360928 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:41.360904    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5"
Sep 14 08:15:41.520074 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:41.519992    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6904f040-1cf1-4f5f-a755-1e24125e03dd-maas-api-service-ca\") pod \"6904f040-1cf1-4f5f-a755-1e24125e03dd\" (UID: \"6904f040-1cf1-4f5f-a755-1e24125e03dd\") "
Sep 14 08:15:41.520431 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:41.520411    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6904f040-1cf1-4f5f-a755-1e24125e03dd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6904f040-1cf1-4f5f-a755-1e24125e03dd" (UID: "6904f040-1cf1-4f5f-a755-1e24125e03dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:15:41.594908 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:41.594876    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:41.621224 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:41.621181    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6904f040-1cf1-4f5f-a755-1e24125e03dd-maas-api-service-ca\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:15:42.229055 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:42.229029    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5"
Sep 14 08:15:42.229503 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:42.229026    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5" event={"ID":"6904f040-1cf1-4f5f-a755-1e24125e03dd","Type":"ContainerDied","Data":"8ea5387a370760ef994bd8d55b30c1292fc0a1d73a68a2fdfeb6b24392160377"}
Sep 14 08:15:42.229503 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:42.229171    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ea5387a370760ef994bd8d55b30c1292fc0a1d73a68a2fdfeb6b24392160377"
Sep 14 08:15:42.378819 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:42.378789    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:43.138669 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.138635    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:43.152886 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.152860    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:15:43.235825 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.235796    2576 generic.go:356] "Generic (PLEG): container finished" podID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerID="6bfaa81ca4a86fa635e1fecb64d22540477f13f6908ef014406415f9f5d3ba31" exitCode=0
Sep 14 08:15:43.236240 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.235848    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"
Sep 14 08:15:43.236240 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.235847    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t" event={"ID":"4b043a3b-f9f4-4084-832a-729fdaa16278","Type":"ContainerDied","Data":"6bfaa81ca4a86fa635e1fecb64d22540477f13f6908ef014406415f9f5d3ba31"}
Sep 14 08:15:43.236240 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.235974    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t" event={"ID":"4b043a3b-f9f4-4084-832a-729fdaa16278","Type":"ContainerDied","Data":"9becfa8c84cec85bac7604b88eaef2c7dc17c9a7f63fca4cf9ed4cfe7813f99c"}
Sep 14 08:15:43.236240 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.236003    2576 scope.go:117] "RemoveContainer" containerID="6bfaa81ca4a86fa635e1fecb64d22540477f13f6908ef014406415f9f5d3ba31"
Sep 14 08:15:43.237016 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.236946    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-workload-certs\") pod \"4b043a3b-f9f4-4084-832a-729fdaa16278\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") "
Sep 14 08:15:43.237016 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.236994    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-token\") pod \"4b043a3b-f9f4-4084-832a-729fdaa16278\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") "
Sep 14 08:15:43.237090 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.237022    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-envoy\") pod \"4b043a3b-f9f4-4084-832a-729fdaa16278\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") "
Sep 14 08:15:43.237090 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.237047    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-workload-socket\") pod \"4b043a3b-f9f4-4084-832a-729fdaa16278\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") "
Sep 14 08:15:43.237180 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.237113    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqs4x\" (UniqueName: \"kubernetes.io/projected/4b043a3b-f9f4-4084-832a-729fdaa16278-kube-api-access-jqs4x\") pod \"4b043a3b-f9f4-4084-832a-729fdaa16278\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") "
Sep 14 08:15:43.237180 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.237140    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-podinfo\") pod \"4b043a3b-f9f4-4084-832a-729fdaa16278\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") "
Sep 14 08:15:43.237310 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.237186    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-data\") pod \"4b043a3b-f9f4-4084-832a-729fdaa16278\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") "
Sep 14 08:15:43.237310 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.237225    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b043a3b-f9f4-4084-832a-729fdaa16278-istiod-ca-cert\") pod \"4b043a3b-f9f4-4084-832a-729fdaa16278\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") "
Sep 14 08:15:43.237310 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.237262    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-credential-socket\") pod \"4b043a3b-f9f4-4084-832a-729fdaa16278\" (UID: \"4b043a3b-f9f4-4084-832a-729fdaa16278\") "
Sep 14 08:15:43.237459 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.237315    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4b043a3b-f9f4-4084-832a-729fdaa16278" (UID: "4b043a3b-f9f4-4084-832a-729fdaa16278"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:15:43.237459 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.237410    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4b043a3b-f9f4-4084-832a-729fdaa16278" (UID: "4b043a3b-f9f4-4084-832a-729fdaa16278"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:15:43.237700 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.237670    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4b043a3b-f9f4-4084-832a-729fdaa16278" (UID: "4b043a3b-f9f4-4084-832a-729fdaa16278"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:15:43.237829 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.237691    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-data" (OuterVolumeSpecName: "istio-data") pod "4b043a3b-f9f4-4084-832a-729fdaa16278" (UID: "4b043a3b-f9f4-4084-832a-729fdaa16278"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:15:43.237829 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.237738    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:15:43.237829 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.237747    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b043a3b-f9f4-4084-832a-729fdaa16278-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4b043a3b-f9f4-4084-832a-729fdaa16278" (UID: "4b043a3b-f9f4-4084-832a-729fdaa16278"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:15:43.237829 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.237763    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:15:43.239542 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.239518    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4b043a3b-f9f4-4084-832a-729fdaa16278" (UID: "4b043a3b-f9f4-4084-832a-729fdaa16278"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:15:43.239850 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.239801    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b043a3b-f9f4-4084-832a-729fdaa16278-kube-api-access-jqs4x" (OuterVolumeSpecName: "kube-api-access-jqs4x") pod "4b043a3b-f9f4-4084-832a-729fdaa16278" (UID: "4b043a3b-f9f4-4084-832a-729fdaa16278"). InnerVolumeSpecName "kube-api-access-jqs4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:15:43.239979 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.239956    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4b043a3b-f9f4-4084-832a-729fdaa16278" (UID: "4b043a3b-f9f4-4084-832a-729fdaa16278"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:15:43.240038 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.240005    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-token" (OuterVolumeSpecName: "istio-token") pod "4b043a3b-f9f4-4084-832a-729fdaa16278" (UID: "4b043a3b-f9f4-4084-832a-729fdaa16278"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:15:43.258812 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.258793    2576 scope.go:117] "RemoveContainer" containerID="93391552eeba8d4e94b2a8974479de6b32e733c76d8200c21999fabf56fde8f0"
Sep 14 08:15:43.267041 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.267021    2576 scope.go:117] "RemoveContainer" containerID="6bfaa81ca4a86fa635e1fecb64d22540477f13f6908ef014406415f9f5d3ba31"
Sep 14 08:15:43.267286 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:15:43.267269    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bfaa81ca4a86fa635e1fecb64d22540477f13f6908ef014406415f9f5d3ba31\": container with ID starting with 6bfaa81ca4a86fa635e1fecb64d22540477f13f6908ef014406415f9f5d3ba31 not found: ID does not exist" containerID="6bfaa81ca4a86fa635e1fecb64d22540477f13f6908ef014406415f9f5d3ba31"
Sep 14 08:15:43.267329 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.267294    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bfaa81ca4a86fa635e1fecb64d22540477f13f6908ef014406415f9f5d3ba31"} err="failed to get container status \"6bfaa81ca4a86fa635e1fecb64d22540477f13f6908ef014406415f9f5d3ba31\": rpc error: code = NotFound desc = could not find container \"6bfaa81ca4a86fa635e1fecb64d22540477f13f6908ef014406415f9f5d3ba31\": container with ID starting with 6bfaa81ca4a86fa635e1fecb64d22540477f13f6908ef014406415f9f5d3ba31 not found: ID does not exist"
Sep 14 08:15:43.267329 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.267309    2576 scope.go:117] "RemoveContainer" containerID="93391552eeba8d4e94b2a8974479de6b32e733c76d8200c21999fabf56fde8f0"
Sep 14 08:15:43.267583 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:15:43.267566    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93391552eeba8d4e94b2a8974479de6b32e733c76d8200c21999fabf56fde8f0\": container with ID starting with 93391552eeba8d4e94b2a8974479de6b32e733c76d8200c21999fabf56fde8f0 not found: ID does not exist" containerID="93391552eeba8d4e94b2a8974479de6b32e733c76d8200c21999fabf56fde8f0"
Sep 14 08:15:43.267625 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.267589    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93391552eeba8d4e94b2a8974479de6b32e733c76d8200c21999fabf56fde8f0"} err="failed to get container status \"93391552eeba8d4e94b2a8974479de6b32e733c76d8200c21999fabf56fde8f0\": rpc error: code = NotFound desc = could not find container \"93391552eeba8d4e94b2a8974479de6b32e733c76d8200c21999fabf56fde8f0\": container with ID starting with 93391552eeba8d4e94b2a8974479de6b32e733c76d8200c21999fabf56fde8f0 not found: ID does not exist"
Sep 14 08:15:43.339259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.339162    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqs4x\" (UniqueName: \"kubernetes.io/projected/4b043a3b-f9f4-4084-832a-729fdaa16278-kube-api-access-jqs4x\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:15:43.339259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.339190    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:15:43.339259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.339200    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:15:43.339259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.339238    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b043a3b-f9f4-4084-832a-729fdaa16278-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:15:43.339259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.339247    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:15:43.339259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.339257    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:15:43.339259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.339265    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b043a3b-f9f4-4084-832a-729fdaa16278-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:15:43.557872 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.557830    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"]
Sep 14 08:15:43.561717 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.561684    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-b22414-openshift-default-7f87cbdcd-znl7t"]
Sep 14 08:15:43.762469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.762441    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:43.841397 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:43.841362    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b043a3b-f9f4-4084-832a-729fdaa16278" path="/var/lib/kubelet/pods/4b043a3b-f9f4-4084-832a-729fdaa16278/volumes"
Sep 14 08:15:48.691756 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:48.691715    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:15:53.862109 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:53.862076    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:53.922654 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:53.922618    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:53.983396 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:53.983366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:54.043019 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:54.042986    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5"]
Sep 14 08:15:54.048918 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:54.048882    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8e721c-htpj5"]
Sep 14 08:15:54.049825 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:54.049795    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:54.110348 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:54.110314    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:54.422423 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:54.422388    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:54.551387 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:54.551352    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:55.206539 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:55.206508    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:55.313008 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:55.312951    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:55.841071 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:55.841027    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6904f040-1cf1-4f5f-a755-1e24125e03dd" path="/var/lib/kubelet/pods/6904f040-1cf1-4f5f-a755-1e24125e03dd/volumes"
Sep 14 08:15:55.953231 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:55.953184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:56.737879 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:56.737839    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:57.920033 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:57.920002    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:58.127174 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:58.127103    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:58.758262 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:58.758234    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:59.509652 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:15:59.509618    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:00.405982 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:00.405951    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:00.541464 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:00.541432    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:00.638452 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:00.638423    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:00.762247 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:00.762144    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"]
Sep 14 08:16:00.762457 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:00.762424    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerName="istio-proxy" containerID="cri-o://b24277a4106c19b2093f2e553eb0ff84af6c6477fb4747dbedfb44018da39fbf" gracePeriod=30
Sep 14 08:16:00.780173 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:00.780151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:01.453084 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:01.453045    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:02.228963 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:02.228929    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:02.749038 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:02.748999    2576 patch_prober.go:28] interesting pod/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 14 08:16:02.749199 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:02.749055    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:16:03.465577 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:03.465543    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:03.738880 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:03.738804    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:04.375970 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:04.375942    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:05.173601 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:05.173569    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:06.007976 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.007949    2576 scope.go:117] "RemoveContainer" containerID="895b7be6acd6dc2b16568c9a7bb6a39c3199eba258ebb8eaf3ea23b5fa6b64ae"
Sep 14 08:16:06.016081 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.016061    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:06.020182 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.020159    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:16:06.056686 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.056658    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-podinfo\") pod \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") "
Sep 14 08:16:06.056809 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.056720    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istiod-ca-cert\") pod \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") "
Sep 14 08:16:06.056809 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.056765    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-credential-socket\") pod \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") "
Sep 14 08:16:06.056809 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.056792    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-token\") pod \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") "
Sep 14 08:16:06.056962 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.056828    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-envoy\") pod \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") "
Sep 14 08:16:06.056962 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.056855    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8zgn\" (UniqueName: \"kubernetes.io/projected/0986547a-8cd0-4d9f-9745-bd65f8ad5085-kube-api-access-n8zgn\") pod \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") "
Sep 14 08:16:06.056962 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.056884    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-workload-socket\") pod \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") "
Sep 14 08:16:06.057117 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.056969    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-workload-certs\") pod \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") "
Sep 14 08:16:06.057117 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.057020    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-data\") pod \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\" (UID: \"0986547a-8cd0-4d9f-9745-bd65f8ad5085\") "
Sep 14 08:16:06.057117 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.057068    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0986547a-8cd0-4d9f-9745-bd65f8ad5085" (UID: "0986547a-8cd0-4d9f-9745-bd65f8ad5085"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:06.057441 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.057412    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0986547a-8cd0-4d9f-9745-bd65f8ad5085" (UID: "0986547a-8cd0-4d9f-9745-bd65f8ad5085"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:06.057523 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.057476    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:06.058011 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.057883    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-data" (OuterVolumeSpecName: "istio-data") pod "0986547a-8cd0-4d9f-9745-bd65f8ad5085" (UID: "0986547a-8cd0-4d9f-9745-bd65f8ad5085"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:06.058011 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.057971    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0986547a-8cd0-4d9f-9745-bd65f8ad5085" (UID: "0986547a-8cd0-4d9f-9745-bd65f8ad5085"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:06.058442 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.058415    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0986547a-8cd0-4d9f-9745-bd65f8ad5085" (UID: "0986547a-8cd0-4d9f-9745-bd65f8ad5085"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:06.059712 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.059664    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-token" (OuterVolumeSpecName: "istio-token") pod "0986547a-8cd0-4d9f-9745-bd65f8ad5085" (UID: "0986547a-8cd0-4d9f-9745-bd65f8ad5085"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:06.059818 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.059767    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0986547a-8cd0-4d9f-9745-bd65f8ad5085-kube-api-access-n8zgn" (OuterVolumeSpecName: "kube-api-access-n8zgn") pod "0986547a-8cd0-4d9f-9745-bd65f8ad5085" (UID: "0986547a-8cd0-4d9f-9745-bd65f8ad5085"). InnerVolumeSpecName "kube-api-access-n8zgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:06.059818 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.059787    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0986547a-8cd0-4d9f-9745-bd65f8ad5085" (UID: "0986547a-8cd0-4d9f-9745-bd65f8ad5085"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:16:06.060711 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.060687    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0986547a-8cd0-4d9f-9745-bd65f8ad5085" (UID: "0986547a-8cd0-4d9f-9745-bd65f8ad5085"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:06.158511 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.158480    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:06.158511 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.158509    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8zgn\" (UniqueName: \"kubernetes.io/projected/0986547a-8cd0-4d9f-9745-bd65f8ad5085-kube-api-access-n8zgn\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:06.158725 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.158524    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:06.158725 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.158536    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:06.158725 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.158548    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:06.158725 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.158559    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:06.158725 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.158568    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:06.158725 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.158580    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0986547a-8cd0-4d9f-9745-bd65f8ad5085-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:06.375939 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.375909    2576 generic.go:356] "Generic (PLEG): container finished" podID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerID="b24277a4106c19b2093f2e553eb0ff84af6c6477fb4747dbedfb44018da39fbf" exitCode=0
Sep 14 08:16:06.376327 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.375955    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"
Sep 14 08:16:06.376327 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.375952    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" event={"ID":"0986547a-8cd0-4d9f-9745-bd65f8ad5085","Type":"ContainerDied","Data":"b24277a4106c19b2093f2e553eb0ff84af6c6477fb4747dbedfb44018da39fbf"}
Sep 14 08:16:06.376327 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.376069    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw" event={"ID":"0986547a-8cd0-4d9f-9745-bd65f8ad5085","Type":"ContainerDied","Data":"b3d33544a6126de31c5d17bc763a5df5a35d33806ac3758ca6b3e2fe4c72b381"}
Sep 14 08:16:06.376327 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.376094    2576 scope.go:117] "RemoveContainer" containerID="b24277a4106c19b2093f2e553eb0ff84af6c6477fb4747dbedfb44018da39fbf"
Sep 14 08:16:06.385064 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.385050    2576 scope.go:117] "RemoveContainer" containerID="6334361d545da6845fe87b97dfa5a3ca3bc24b920e96614bbdcfa035da77c8c9"
Sep 14 08:16:06.393037 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.393018    2576 scope.go:117] "RemoveContainer" containerID="b24277a4106c19b2093f2e553eb0ff84af6c6477fb4747dbedfb44018da39fbf"
Sep 14 08:16:06.393301 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:16:06.393281    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b24277a4106c19b2093f2e553eb0ff84af6c6477fb4747dbedfb44018da39fbf\": container with ID starting with b24277a4106c19b2093f2e553eb0ff84af6c6477fb4747dbedfb44018da39fbf not found: ID does not exist" containerID="b24277a4106c19b2093f2e553eb0ff84af6c6477fb4747dbedfb44018da39fbf"
Sep 14 08:16:06.393363 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.393309    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b24277a4106c19b2093f2e553eb0ff84af6c6477fb4747dbedfb44018da39fbf"} err="failed to get container status \"b24277a4106c19b2093f2e553eb0ff84af6c6477fb4747dbedfb44018da39fbf\": rpc error: code = NotFound desc = could not find container \"b24277a4106c19b2093f2e553eb0ff84af6c6477fb4747dbedfb44018da39fbf\": container with ID starting with b24277a4106c19b2093f2e553eb0ff84af6c6477fb4747dbedfb44018da39fbf not found: ID does not exist"
Sep 14 08:16:06.393363 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.393326    2576 scope.go:117] "RemoveContainer" containerID="6334361d545da6845fe87b97dfa5a3ca3bc24b920e96614bbdcfa035da77c8c9"
Sep 14 08:16:06.393560 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:16:06.393543    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6334361d545da6845fe87b97dfa5a3ca3bc24b920e96614bbdcfa035da77c8c9\": container with ID starting with 6334361d545da6845fe87b97dfa5a3ca3bc24b920e96614bbdcfa035da77c8c9 not found: ID does not exist" containerID="6334361d545da6845fe87b97dfa5a3ca3bc24b920e96614bbdcfa035da77c8c9"
Sep 14 08:16:06.393613 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.393570    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6334361d545da6845fe87b97dfa5a3ca3bc24b920e96614bbdcfa035da77c8c9"} err="failed to get container status \"6334361d545da6845fe87b97dfa5a3ca3bc24b920e96614bbdcfa035da77c8c9\": rpc error: code = NotFound desc = could not find container \"6334361d545da6845fe87b97dfa5a3ca3bc24b920e96614bbdcfa035da77c8c9\": container with ID starting with 6334361d545da6845fe87b97dfa5a3ca3bc24b920e96614bbdcfa035da77c8c9 not found: ID does not exist"
Sep 14 08:16:06.397796 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.397775    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"]
Sep 14 08:16:06.401271 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:06.401251    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-8e721c-openshift-default-58847ddb54-rxvqw"]
Sep 14 08:16:07.838930 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:07.838894    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" path="/var/lib/kubelet/pods/0986547a-8cd0-4d9f-9745-bd65f8ad5085/volumes"
Sep 14 08:16:11.643976 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:11.643946    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:11.649384 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:11.649354    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd7582-29822895-9kbdx"]
Sep 14 08:16:11.651414 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:11.651393    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd7582-29822895-9kbdx"]
Sep 14 08:16:11.794709 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:11.794677    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0b8346-29822895-b7nrg"]
Sep 14 08:16:11.797542 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:11.797517    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0b8346-29822895-b7nrg"]
Sep 14 08:16:11.841079 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:11.841049    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70ee8f1e-8ea8-4646-91d7-66f8c00a6366" path="/var/lib/kubelet/pods/70ee8f1e-8ea8-4646-91d7-66f8c00a6366/volumes"
Sep 14 08:16:11.841384 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:11.841364    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d779413a-6ffd-4915-b201-ba4e97923d8f" path="/var/lib/kubelet/pods/d779413a-6ffd-4915-b201-ba4e97923d8f/volumes"
Sep 14 08:16:11.925906 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:11.925824    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:12.003938 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:12.003895    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:12.120476 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:12.120417    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:12.218795 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:12.218682    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:12.317130 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:12.317061    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:12.418500 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:12.418467    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:12.554129 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:12.554028    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:13.031485 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:13.031449    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:13.659876 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:13.659827    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:14.390045 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:14.390007    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:16:14.442735 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:14.442711    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:15.099221 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:15.099170    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:18.599966 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:18.599937    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:18.716626 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:18.716595    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:18.934322 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:18.934290    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"]
Sep 14 08:16:18.934577 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:18.934548    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy" containerID="cri-o://ff8eba2cb98b92d9fdd6ce618f8959810fbd6110b4b4041a853ed0ba552911fe" gracePeriod=30
Sep 14 08:16:18.983476 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:18.983451    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:19.505095 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:19.505058    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:19.605813 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:19.605783    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:19.678302 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:19.678251    2576 patch_prober.go:28] interesting pod/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 08:16:19.678463 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:19.678329    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:16:19.805416 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:19.805337    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:20.468488 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:20.468456    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:21.240262 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:21.240234    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:21.869775 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:21.869741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:22.510325 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:22.510290    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:23.253025 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:23.252995    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:23.765886 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:23.765841    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"]
Sep 14 08:16:23.783173 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:23.783143    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:24.183120 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.183086    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:16:24.335629 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.335557    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m86p8\" (UniqueName: \"kubernetes.io/projected/db74fb53-47cb-4e85-9aab-c7feadb5f64f-kube-api-access-m86p8\") pod \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") "
Sep 14 08:16:24.335629 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.335609    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-workload-socket\") pod \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") "
Sep 14 08:16:24.335828 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.335653    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-token\") pod \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") "
Sep 14 08:16:24.335828 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.335677    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-podinfo\") pod \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") "
Sep 14 08:16:24.335937 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.335822    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-credential-socket\") pod \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") "
Sep 14 08:16:24.335937 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.335875    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-data\") pod \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") "
Sep 14 08:16:24.335937 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.335905    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-workload-certs\") pod \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") "
Sep 14 08:16:24.336094 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.335940    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-envoy\") pod \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") "
Sep 14 08:16:24.336094 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.335964    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "db74fb53-47cb-4e85-9aab-c7feadb5f64f" (UID: "db74fb53-47cb-4e85-9aab-c7feadb5f64f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:24.336094 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.336009    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istiod-ca-cert\") pod \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\" (UID: \"db74fb53-47cb-4e85-9aab-c7feadb5f64f\") "
Sep 14 08:16:24.336303 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.336096    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "db74fb53-47cb-4e85-9aab-c7feadb5f64f" (UID: "db74fb53-47cb-4e85-9aab-c7feadb5f64f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:24.336303 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.336158    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "db74fb53-47cb-4e85-9aab-c7feadb5f64f" (UID: "db74fb53-47cb-4e85-9aab-c7feadb5f64f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:24.336303 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.336275    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-data" (OuterVolumeSpecName: "istio-data") pod "db74fb53-47cb-4e85-9aab-c7feadb5f64f" (UID: "db74fb53-47cb-4e85-9aab-c7feadb5f64f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:24.336468 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.336434    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:24.336468 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.336454    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:24.336542 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.336470    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:24.336542 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.336483    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:24.336542 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.336481    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "db74fb53-47cb-4e85-9aab-c7feadb5f64f" (UID: "db74fb53-47cb-4e85-9aab-c7feadb5f64f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:24.337918 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.337890    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-token" (OuterVolumeSpecName: "istio-token") pod "db74fb53-47cb-4e85-9aab-c7feadb5f64f" (UID: "db74fb53-47cb-4e85-9aab-c7feadb5f64f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:24.338259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.338236    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db74fb53-47cb-4e85-9aab-c7feadb5f64f-kube-api-access-m86p8" (OuterVolumeSpecName: "kube-api-access-m86p8") pod "db74fb53-47cb-4e85-9aab-c7feadb5f64f" (UID: "db74fb53-47cb-4e85-9aab-c7feadb5f64f"). InnerVolumeSpecName "kube-api-access-m86p8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:24.338348 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.338257    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "db74fb53-47cb-4e85-9aab-c7feadb5f64f" (UID: "db74fb53-47cb-4e85-9aab-c7feadb5f64f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:16:24.338431 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.338414    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "db74fb53-47cb-4e85-9aab-c7feadb5f64f" (UID: "db74fb53-47cb-4e85-9aab-c7feadb5f64f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:24.437542 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.437518    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m86p8\" (UniqueName: \"kubernetes.io/projected/db74fb53-47cb-4e85-9aab-c7feadb5f64f-kube-api-access-m86p8\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:24.437542 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.437540    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:24.437671 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.437549    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:24.437671 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.437557    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:24.437671 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.437565    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db74fb53-47cb-4e85-9aab-c7feadb5f64f-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:24.480756 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.480725    2576 generic.go:356] "Generic (PLEG): container finished" podID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerID="ff8eba2cb98b92d9fdd6ce618f8959810fbd6110b4b4041a853ed0ba552911fe" exitCode=0
Sep 14 08:16:24.480894 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.480776    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"
Sep 14 08:16:24.480894 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.480809    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" event={"ID":"db74fb53-47cb-4e85-9aab-c7feadb5f64f","Type":"ContainerDied","Data":"ff8eba2cb98b92d9fdd6ce618f8959810fbd6110b4b4041a853ed0ba552911fe"}
Sep 14 08:16:24.480894 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.480865    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst" event={"ID":"db74fb53-47cb-4e85-9aab-c7feadb5f64f","Type":"ContainerDied","Data":"c345281a4b406a647d07a019e66e58367c721db9665603b02d2e1f533cef7b5b"}
Sep 14 08:16:24.480894 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.480890    2576 scope.go:117] "RemoveContainer" containerID="ff8eba2cb98b92d9fdd6ce618f8959810fbd6110b4b4041a853ed0ba552911fe"
Sep 14 08:16:24.493277 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.493258    2576 scope.go:117] "RemoveContainer" containerID="3d4693a3c0931d55e70ea78e00e155c4e4e605566a6775bb4284320ee2a631ea"
Sep 14 08:16:24.501998 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.501980    2576 scope.go:117] "RemoveContainer" containerID="ff8eba2cb98b92d9fdd6ce618f8959810fbd6110b4b4041a853ed0ba552911fe"
Sep 14 08:16:24.502390 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:16:24.502369    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff8eba2cb98b92d9fdd6ce618f8959810fbd6110b4b4041a853ed0ba552911fe\": container with ID starting with ff8eba2cb98b92d9fdd6ce618f8959810fbd6110b4b4041a853ed0ba552911fe not found: ID does not exist" containerID="ff8eba2cb98b92d9fdd6ce618f8959810fbd6110b4b4041a853ed0ba552911fe"
Sep 14 08:16:24.502483 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.502400    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8eba2cb98b92d9fdd6ce618f8959810fbd6110b4b4041a853ed0ba552911fe"} err="failed to get container status \"ff8eba2cb98b92d9fdd6ce618f8959810fbd6110b4b4041a853ed0ba552911fe\": rpc error: code = NotFound desc = could not find container \"ff8eba2cb98b92d9fdd6ce618f8959810fbd6110b4b4041a853ed0ba552911fe\": container with ID starting with ff8eba2cb98b92d9fdd6ce618f8959810fbd6110b4b4041a853ed0ba552911fe not found: ID does not exist"
Sep 14 08:16:24.502483 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.502425    2576 scope.go:117] "RemoveContainer" containerID="3d4693a3c0931d55e70ea78e00e155c4e4e605566a6775bb4284320ee2a631ea"
Sep 14 08:16:24.503016 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:16:24.502994    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d4693a3c0931d55e70ea78e00e155c4e4e605566a6775bb4284320ee2a631ea\": container with ID starting with 3d4693a3c0931d55e70ea78e00e155c4e4e605566a6775bb4284320ee2a631ea not found: ID does not exist" containerID="3d4693a3c0931d55e70ea78e00e155c4e4e605566a6775bb4284320ee2a631ea"
Sep 14 08:16:24.503126 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.503025    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4693a3c0931d55e70ea78e00e155c4e4e605566a6775bb4284320ee2a631ea"} err="failed to get container status \"3d4693a3c0931d55e70ea78e00e155c4e4e605566a6775bb4284320ee2a631ea\": rpc error: code = NotFound desc = could not find container \"3d4693a3c0931d55e70ea78e00e155c4e4e605566a6775bb4284320ee2a631ea\": container with ID starting with 3d4693a3c0931d55e70ea78e00e155c4e4e605566a6775bb4284320ee2a631ea not found: ID does not exist"
Sep 14 08:16:24.505225 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.505190    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"]
Sep 14 08:16:24.507186 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:24.507165    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-cd7582-openshift-default-7d574b8f58-9wvst"]
Sep 14 08:16:25.818222 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:25.818172    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:25.839551 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:25.839525    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" path="/var/lib/kubelet/pods/db74fb53-47cb-4e85-9aab-c7feadb5f64f/volumes"
Sep 14 08:16:30.630703 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:30.630669    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:30.692383 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:30.692346    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:30.752841 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:30.752810    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:30.815193 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:30.815169    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:30.870286 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:30.870266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:31.202574 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:31.202540    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:31.303415 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:31.303379    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:31.932602 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:31.932568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:32.572841 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:32.572811    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:33.999760 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:33.999731    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:16:36.497577 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:36.497543    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:36.625194 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:36.625162    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:38.168703 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:38.168646    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:38.458668 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:38.458582    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:38.575398 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:38.575368    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:38.730100 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:38.730022    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"]
Sep 14 08:16:38.730323 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:38.730297    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2" podUID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerName="istio-proxy" containerID="cri-o://4d6ea8b503ca48bf8db2be961870377f788c5be0a74db33ef76fff1b413bf6f1" gracePeriod=30
Sep 14 08:16:38.754328 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:38.754306    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:38.786494 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:38.786461    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb" podUID="46858da9-0ca0-4607-b32c-df0351851164" containerName="main" containerID="cri-o://72ef10d6e52f48a4df90bb951576dd173ac8ae24608be58c95cde39794a343d9" gracePeriod=45
Sep 14 08:16:39.039528 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.039505    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:16:39.083864 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.083834    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-model-cache\") pod \"46858da9-0ca0-4607-b32c-df0351851164\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") "
Sep 14 08:16:39.084006 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.083916    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-dshm\") pod \"46858da9-0ca0-4607-b32c-df0351851164\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") "
Sep 14 08:16:39.084006 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.083948    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-home\") pod \"46858da9-0ca0-4607-b32c-df0351851164\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") "
Sep 14 08:16:39.084006 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.083981    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-tmp-dir\") pod \"46858da9-0ca0-4607-b32c-df0351851164\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") "
Sep 14 08:16:39.084172 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.084018    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46858da9-0ca0-4607-b32c-df0351851164-tls-certs\") pod \"46858da9-0ca0-4607-b32c-df0351851164\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") "
Sep 14 08:16:39.084172 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.084064    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbq5z\" (UniqueName: \"kubernetes.io/projected/46858da9-0ca0-4607-b32c-df0351851164-kube-api-access-xbq5z\") pod \"46858da9-0ca0-4607-b32c-df0351851164\" (UID: \"46858da9-0ca0-4607-b32c-df0351851164\") "
Sep 14 08:16:39.084172 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.084094    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-model-cache" (OuterVolumeSpecName: "model-cache") pod "46858da9-0ca0-4607-b32c-df0351851164" (UID: "46858da9-0ca0-4607-b32c-df0351851164"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:39.084366 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.084176    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-home" (OuterVolumeSpecName: "home") pod "46858da9-0ca0-4607-b32c-df0351851164" (UID: "46858da9-0ca0-4607-b32c-df0351851164"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:39.084366 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.084287    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "46858da9-0ca0-4607-b32c-df0351851164" (UID: "46858da9-0ca0-4607-b32c-df0351851164"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:39.084512 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.084486    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-model-cache\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:39.084575 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.084511    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-home\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:39.084575 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.084525    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-tmp-dir\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:39.086498 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.086466    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-dshm" (OuterVolumeSpecName: "dshm") pod "46858da9-0ca0-4607-b32c-df0351851164" (UID: "46858da9-0ca0-4607-b32c-df0351851164"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:39.086498 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.086484    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46858da9-0ca0-4607-b32c-df0351851164-kube-api-access-xbq5z" (OuterVolumeSpecName: "kube-api-access-xbq5z") pod "46858da9-0ca0-4607-b32c-df0351851164" (UID: "46858da9-0ca0-4607-b32c-df0351851164"). InnerVolumeSpecName "kube-api-access-xbq5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:39.086691 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.086545    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46858da9-0ca0-4607-b32c-df0351851164-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "46858da9-0ca0-4607-b32c-df0351851164" (UID: "46858da9-0ca0-4607-b32c-df0351851164"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:16:39.185503 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.185481    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46858da9-0ca0-4607-b32c-df0351851164-dshm\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:39.185503 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.185503    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46858da9-0ca0-4607-b32c-df0351851164-tls-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:39.185813 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.185514    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbq5z\" (UniqueName: \"kubernetes.io/projected/46858da9-0ca0-4607-b32c-df0351851164-kube-api-access-xbq5z\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:39.561411 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.561379    2576 generic.go:356] "Generic (PLEG): container finished" podID="46858da9-0ca0-4607-b32c-df0351851164" containerID="72ef10d6e52f48a4df90bb951576dd173ac8ae24608be58c95cde39794a343d9" exitCode=0
Sep 14 08:16:39.561541 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.561423    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"
Sep 14 08:16:39.561541 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.561467    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb" event={"ID":"46858da9-0ca0-4607-b32c-df0351851164","Type":"ContainerDied","Data":"72ef10d6e52f48a4df90bb951576dd173ac8ae24608be58c95cde39794a343d9"}
Sep 14 08:16:39.561541 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.561513    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb" event={"ID":"46858da9-0ca0-4607-b32c-df0351851164","Type":"ContainerDied","Data":"1bd91cf676645b56b847c4b003d17e995e7bcb79240bbc3f509b1e19d2fe3e13"}
Sep 14 08:16:39.561541 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.561530    2576 scope.go:117] "RemoveContainer" containerID="72ef10d6e52f48a4df90bb951576dd173ac8ae24608be58c95cde39794a343d9"
Sep 14 08:16:39.571139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.571120    2576 scope.go:117] "RemoveContainer" containerID="72ef10d6e52f48a4df90bb951576dd173ac8ae24608be58c95cde39794a343d9"
Sep 14 08:16:39.571450 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:16:39.571429    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72ef10d6e52f48a4df90bb951576dd173ac8ae24608be58c95cde39794a343d9\": container with ID starting with 72ef10d6e52f48a4df90bb951576dd173ac8ae24608be58c95cde39794a343d9 not found: ID does not exist" containerID="72ef10d6e52f48a4df90bb951576dd173ac8ae24608be58c95cde39794a343d9"
Sep 14 08:16:39.571515 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.571458    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72ef10d6e52f48a4df90bb951576dd173ac8ae24608be58c95cde39794a343d9"} err="failed to get container status \"72ef10d6e52f48a4df90bb951576dd173ac8ae24608be58c95cde39794a343d9\": rpc error: code = NotFound desc = could not find container \"72ef10d6e52f48a4df90bb951576dd173ac8ae24608be58c95cde39794a343d9\": container with ID starting with 72ef10d6e52f48a4df90bb951576dd173ac8ae24608be58c95cde39794a343d9 not found: ID does not exist"
Sep 14 08:16:39.582845 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.582824    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"]
Sep 14 08:16:39.586107 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.586084    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/auth-test-model-919cc9-kserve-7b4ddd77bc-778sb"]
Sep 14 08:16:39.840514 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:39.840423    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46858da9-0ca0-4607-b32c-df0351851164" path="/var/lib/kubelet/pods/46858da9-0ca0-4607-b32c-df0351851164/volumes"
Sep 14 08:16:42.011060 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:42.011028    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:43.985774 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:43.985752    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:16:44.028368 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.028283    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-data\") pod \"c303423d-4fd4-4871-a852-acae8fb0e1f8\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") "
Sep 14 08:16:44.028368 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.028335    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-workload-certs\") pod \"c303423d-4fd4-4871-a852-acae8fb0e1f8\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") "
Sep 14 08:16:44.028663 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.028387    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-envoy\") pod \"c303423d-4fd4-4871-a852-acae8fb0e1f8\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") "
Sep 14 08:16:44.028663 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.028414    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c303423d-4fd4-4871-a852-acae8fb0e1f8-istiod-ca-cert\") pod \"c303423d-4fd4-4871-a852-acae8fb0e1f8\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") "
Sep 14 08:16:44.028663 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.028442    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-token\") pod \"c303423d-4fd4-4871-a852-acae8fb0e1f8\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") "
Sep 14 08:16:44.028663 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.028469    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-podinfo\") pod \"c303423d-4fd4-4871-a852-acae8fb0e1f8\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") "
Sep 14 08:16:44.028663 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.028495    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-workload-socket\") pod \"c303423d-4fd4-4871-a852-acae8fb0e1f8\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") "
Sep 14 08:16:44.028663 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.028527    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-credential-socket\") pod \"c303423d-4fd4-4871-a852-acae8fb0e1f8\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") "
Sep 14 08:16:44.028663 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.028568    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c897f\" (UniqueName: \"kubernetes.io/projected/c303423d-4fd4-4871-a852-acae8fb0e1f8-kube-api-access-c897f\") pod \"c303423d-4fd4-4871-a852-acae8fb0e1f8\" (UID: \"c303423d-4fd4-4871-a852-acae8fb0e1f8\") "
Sep 14 08:16:44.029029 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.028642    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "c303423d-4fd4-4871-a852-acae8fb0e1f8" (UID: "c303423d-4fd4-4871-a852-acae8fb0e1f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:44.029029 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.028711    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c303423d-4fd4-4871-a852-acae8fb0e1f8" (UID: "c303423d-4fd4-4871-a852-acae8fb0e1f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:44.029029 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.028888    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c303423d-4fd4-4871-a852-acae8fb0e1f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c303423d-4fd4-4871-a852-acae8fb0e1f8" (UID: "c303423d-4fd4-4871-a852-acae8fb0e1f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:44.029192 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.029047    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:44.029192 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.029066    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:44.029192 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.029079    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c303423d-4fd4-4871-a852-acae8fb0e1f8-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:44.029192 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.029138    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c303423d-4fd4-4871-a852-acae8fb0e1f8" (UID: "c303423d-4fd4-4871-a852-acae8fb0e1f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:44.029430 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.029392    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c303423d-4fd4-4871-a852-acae8fb0e1f8" (UID: "c303423d-4fd4-4871-a852-acae8fb0e1f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:44.031418 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.031388    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c303423d-4fd4-4871-a852-acae8fb0e1f8" (UID: "c303423d-4fd4-4871-a852-acae8fb0e1f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:44.031560 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.031409    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "c303423d-4fd4-4871-a852-acae8fb0e1f8" (UID: "c303423d-4fd4-4871-a852-acae8fb0e1f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:44.031560 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.031430    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c303423d-4fd4-4871-a852-acae8fb0e1f8-kube-api-access-c897f" (OuterVolumeSpecName: "kube-api-access-c897f") pod "c303423d-4fd4-4871-a852-acae8fb0e1f8" (UID: "c303423d-4fd4-4871-a852-acae8fb0e1f8"). InnerVolumeSpecName "kube-api-access-c897f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:44.031560 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.031464    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c303423d-4fd4-4871-a852-acae8fb0e1f8" (UID: "c303423d-4fd4-4871-a852-acae8fb0e1f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:16:44.130184 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.130149    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c897f\" (UniqueName: \"kubernetes.io/projected/c303423d-4fd4-4871-a852-acae8fb0e1f8-kube-api-access-c897f\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:44.130184 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.130179    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:44.130184 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.130189    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:44.130456 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.130198    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c303423d-4fd4-4871-a852-acae8fb0e1f8-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:44.130456 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.130220    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:44.130456 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.130229    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c303423d-4fd4-4871-a852-acae8fb0e1f8-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:16:44.589521 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.589489    2576 generic.go:356] "Generic (PLEG): container finished" podID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerID="4d6ea8b503ca48bf8db2be961870377f788c5be0a74db33ef76fff1b413bf6f1" exitCode=0
Sep 14 08:16:44.589679 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.589530    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2" event={"ID":"c303423d-4fd4-4871-a852-acae8fb0e1f8","Type":"ContainerDied","Data":"4d6ea8b503ca48bf8db2be961870377f788c5be0a74db33ef76fff1b413bf6f1"}
Sep 14 08:16:44.589679 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.589575    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2" event={"ID":"c303423d-4fd4-4871-a852-acae8fb0e1f8","Type":"ContainerDied","Data":"8b415a0dac6398e89a05fb0d1fc806d1b4f8ed7091d887c641f46513ee3823cd"}
Sep 14 08:16:44.589679 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.589595    2576 scope.go:117] "RemoveContainer" containerID="4d6ea8b503ca48bf8db2be961870377f788c5be0a74db33ef76fff1b413bf6f1"
Sep 14 08:16:44.589679 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.589536    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"
Sep 14 08:16:44.598859 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.598841    2576 scope.go:117] "RemoveContainer" containerID="5efc45ead7fc41f84be1925f07ad65a7976dcab322f1075f3e7af5d300db826b"
Sep 14 08:16:44.606532 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.606516    2576 scope.go:117] "RemoveContainer" containerID="4d6ea8b503ca48bf8db2be961870377f788c5be0a74db33ef76fff1b413bf6f1"
Sep 14 08:16:44.606772 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:16:44.606752    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d6ea8b503ca48bf8db2be961870377f788c5be0a74db33ef76fff1b413bf6f1\": container with ID starting with 4d6ea8b503ca48bf8db2be961870377f788c5be0a74db33ef76fff1b413bf6f1 not found: ID does not exist" containerID="4d6ea8b503ca48bf8db2be961870377f788c5be0a74db33ef76fff1b413bf6f1"
Sep 14 08:16:44.606811 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.606779    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d6ea8b503ca48bf8db2be961870377f788c5be0a74db33ef76fff1b413bf6f1"} err="failed to get container status \"4d6ea8b503ca48bf8db2be961870377f788c5be0a74db33ef76fff1b413bf6f1\": rpc error: code = NotFound desc = could not find container \"4d6ea8b503ca48bf8db2be961870377f788c5be0a74db33ef76fff1b413bf6f1\": container with ID starting with 4d6ea8b503ca48bf8db2be961870377f788c5be0a74db33ef76fff1b413bf6f1 not found: ID does not exist"
Sep 14 08:16:44.606811 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.606795    2576 scope.go:117] "RemoveContainer" containerID="5efc45ead7fc41f84be1925f07ad65a7976dcab322f1075f3e7af5d300db826b"
Sep 14 08:16:44.607035 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:16:44.607017    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5efc45ead7fc41f84be1925f07ad65a7976dcab322f1075f3e7af5d300db826b\": container with ID starting with 5efc45ead7fc41f84be1925f07ad65a7976dcab322f1075f3e7af5d300db826b not found: ID does not exist" containerID="5efc45ead7fc41f84be1925f07ad65a7976dcab322f1075f3e7af5d300db826b"
Sep 14 08:16:44.607072 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.607041    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5efc45ead7fc41f84be1925f07ad65a7976dcab322f1075f3e7af5d300db826b"} err="failed to get container status \"5efc45ead7fc41f84be1925f07ad65a7976dcab322f1075f3e7af5d300db826b\": rpc error: code = NotFound desc = could not find container \"5efc45ead7fc41f84be1925f07ad65a7976dcab322f1075f3e7af5d300db826b\": container with ID starting with 5efc45ead7fc41f84be1925f07ad65a7976dcab322f1075f3e7af5d300db826b not found: ID does not exist"
Sep 14 08:16:44.615049 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.615026    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"]
Sep 14 08:16:44.618541 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:44.618523    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4d8f28-openshift-default-558d7d9d8-rtbs2"]
Sep 14 08:16:45.838741 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:45.838703    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c303423d-4fd4-4871-a852-acae8fb0e1f8" path="/var/lib/kubelet/pods/c303423d-4fd4-4871-a852-acae8fb0e1f8/volumes"
Sep 14 08:16:46.393065 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:46.393037    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:16:52.458772 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:16:52.458738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:17:05.316872 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:17:05.316842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:17:08.901047 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:17:08.901005    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:17:44.596977 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:17:44.596933    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:17:57.190802 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:17:57.190768    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:18:05.071343 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:18:05.071310    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:05.879756 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:18:05.879731    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:07.791319 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:18:07.791287    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:18:08.675637 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:18:08.675603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:09.102828 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:18:09.102747    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:17.995877 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:18:17.995830    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:18:39.711228 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:18:39.711165    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:43.297385 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:18:43.297347    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:19:04.188959 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:04.188925    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:19:14.497945 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:14.497858    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:19:19.826055 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:19.826020    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:20.814681 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:20.814645    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:21.280096 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:21.280061    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:21.600502 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:21.600420    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:22.085952 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:22.085920    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:24.103907 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:24.103873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:24.849470 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:24.849427    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:24.947914 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:24.947881    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:25.730121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:25.730088    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:26.621515 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.621485    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"]
Sep 14 08:19:26.621858 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.621845    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerName="istio-proxy"
Sep 14 08:19:26.621903 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.621860    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerName="istio-proxy"
Sep 14 08:19:26.621903 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.621868    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy"
Sep 14 08:19:26.621903 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.621874    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy"
Sep 14 08:19:26.621903 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.621886    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerName="istio-proxy"
Sep 14 08:19:26.621903 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.621891    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerName="istio-proxy"
Sep 14 08:19:26.621903 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.621898    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46858da9-0ca0-4607-b32c-df0351851164" containerName="main"
Sep 14 08:19:26.621903 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.621903    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="46858da9-0ca0-4607-b32c-df0351851164" containerName="main"
Sep 14 08:19:26.622121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.621922    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerName="istio-proxy"
Sep 14 08:19:26.622121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.621928    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerName="istio-proxy"
Sep 14 08:19:26.622121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.621934    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6904f040-1cf1-4f5f-a755-1e24125e03dd" containerName="revoke-keys"
Sep 14 08:19:26.622121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.621939    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6904f040-1cf1-4f5f-a755-1e24125e03dd" containerName="revoke-keys"
Sep 14 08:19:26.622121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.621993    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy"
Sep 14 08:19:26.622121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.622001    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerName="istio-proxy"
Sep 14 08:19:26.622121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.622006    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerName="istio-proxy"
Sep 14 08:19:26.622121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.622014    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerName="istio-proxy"
Sep 14 08:19:26.622121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.622021    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6904f040-1cf1-4f5f-a755-1e24125e03dd" containerName="revoke-keys"
Sep 14 08:19:26.622121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.622027    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="46858da9-0ca0-4607-b32c-df0351851164" containerName="main"
Sep 14 08:19:26.625466 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.625449    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:26.628282 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.628260    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-03d6fa-serving-cert\""
Sep 14 08:19:26.643337 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.643314    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"]
Sep 14 08:19:26.695628 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.695584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:26.759345 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.759310    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-metrics-certs\") pod \"maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:26.759710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.759362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2qm8\" (UniqueName: \"kubernetes.io/projected/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-kube-api-access-p2qm8\") pod \"maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:26.759710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.759408    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:26.759710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.759468    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-maas-api-tls\") pod \"maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:26.811381 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.811356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:26.860308 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.860288    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-maas-api-tls\") pod \"maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:26.860434 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.860366    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-metrics-certs\") pod \"maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:26.860434 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.860404    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2qm8\" (UniqueName: \"kubernetes.io/projected/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-kube-api-access-p2qm8\") pod \"maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:26.860434 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.860422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:26.861000 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.860975    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:26.862732 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.862712    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-metrics-certs\") pod \"maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:26.862840 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.862786    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-maas-api-tls\") pod \"maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:26.867953 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.867931    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2qm8\" (UniqueName: \"kubernetes.io/projected/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-kube-api-access-p2qm8\") pod \"maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:26.912730 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.912662    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:26.940231 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:26.940175    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:27.190402 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:27.190321    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:27.216335 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:27.216292    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"]
Sep 14 08:19:27.222440 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:27.222407    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:19:27.286037 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:27.286006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:27.421325 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:27.421284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z" event={"ID":"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65","Type":"ContainerStarted","Data":"1b2b1c19723d90878ab061ac8413a6248235a8c195ef30ae36f7478a4e721229"}
Sep 14 08:19:27.586183 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:27.586102    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:19:28.429005 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:28.428972    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z" event={"ID":"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65","Type":"ContainerStarted","Data":"746beb3854d2dba7f566eb7a365dc7ba4f97955acf2d0f9434ac84fc4e586134"}
Sep 14 08:19:28.429406 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:28.429048    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:28.446008 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:28.445957    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z" podStartSLOduration=1.833173816 podStartE2EDuration="2.445941952s" totalImagesPullingTime="612.768136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:19:28.44459571 +0000 UTC m=+1823.149134214" watchObservedRunningTime="2026-09-14 08:19:28.445941952 +0000 UTC m=+1823.150480486"
Sep 14 08:19:34.439334 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:34.439301    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:19:37.697461 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:37.697427    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:19:48.531591 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:48.531552    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:50.038646 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.038611    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:50.581685 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.581648    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:50.821157 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.821116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:50.869827 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.869788    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"]
Sep 14 08:19:50.870377 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.870360    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerName="istio-proxy"
Sep 14 08:19:50.870479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.870380    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerName="istio-proxy"
Sep 14 08:19:50.870479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.870406    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerName="istio-proxy"
Sep 14 08:19:50.870479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.870415    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerName="istio-proxy"
Sep 14 08:19:50.870479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.870425    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy"
Sep 14 08:19:50.870479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.870433    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy"
Sep 14 08:19:50.870479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.870451    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerName="istio-proxy"
Sep 14 08:19:50.870479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.870459    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerName="istio-proxy"
Sep 14 08:19:50.870827 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.870553    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="db74fb53-47cb-4e85-9aab-c7feadb5f64f" containerName="istio-proxy"
Sep 14 08:19:50.870827 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.870565    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b043a3b-f9f4-4084-832a-729fdaa16278" containerName="istio-proxy"
Sep 14 08:19:50.870827 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.870577    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c303423d-4fd4-4871-a852-acae8fb0e1f8" containerName="istio-proxy"
Sep 14 08:19:50.870827 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.870587    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0986547a-8cd0-4d9f-9745-bd65f8ad5085" containerName="istio-proxy"
Sep 14 08:19:50.874017 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.873995    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:50.876388 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.876361    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-7bcf7b-openshift-default-dockercfg-pqwbf\""
Sep 14 08:19:50.883012 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.882988    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"]
Sep 14 08:19:50.910904 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.910882    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:50.971790 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.971277    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-token\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:50.971790 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.971334    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-podinfo\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:50.971790 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.971390    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-data\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:50.971790 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.971421    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istiod-ca-cert\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:50.971790 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.971477    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-workload-certs\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:50.971790 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.971516    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-credential-socket\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:50.971790 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.971563    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-envoy\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:50.971790 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.971646    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-workload-socket\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:50.971790 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:50.971671    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k259\" (UniqueName: \"kubernetes.io/projected/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-kube-api-access-2k259\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.046326 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.046271    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:51.073093 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.073067    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-workload-certs\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.073245 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.073107    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-credential-socket\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.073245 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.073137    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-envoy\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.073321 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.073278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-workload-socket\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.073321 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.073314    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k259\" (UniqueName: \"kubernetes.io/projected/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-kube-api-access-2k259\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.073399 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.073355    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-token\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.073399 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.073383    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-podinfo\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.073503 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.073393    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-workload-certs\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.073503 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.073454    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-data\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.073503 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.073487    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istiod-ca-cert\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.073661 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.073513    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-credential-socket\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.073661 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.073598    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-workload-socket\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.073853 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.073831    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-data\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.074198 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.074175    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istiod-ca-cert\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.075523 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.075502    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-envoy\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.075848 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.075826    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-podinfo\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.080918 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.080893    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-token\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.081036 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.081022    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k259\" (UniqueName: \"kubernetes.io/projected/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-kube-api-access-2k259\") pod \"e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") " pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.185898 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.185817    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:51.464793 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.464764    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"]
Sep 14 08:19:51.466964 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:19:51.466937    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbdc18a7_9bef_4859_8017_6b9b3c3ad5de.slice/crio-d564d11c8f0df8c2766979276a32644bc0b66492324716d3966919c1ca369016 WatchSource:0}: Error finding container d564d11c8f0df8c2766979276a32644bc0b66492324716d3966919c1ca369016: Status 404 returned error can't find the container with id d564d11c8f0df8c2766979276a32644bc0b66492324716d3966919c1ca369016
Sep 14 08:19:51.469188 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.469153    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:19:51.469259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.469235    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:19:51.469298 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.469263    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:19:51.561301 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.561258    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn" event={"ID":"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de","Type":"ContainerStarted","Data":"85c60f8ea2561006f1a9e14c7071b380d9e7003bf1dab70d79743af53d969c3e"}
Sep 14 08:19:51.561468 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.561337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn" event={"ID":"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de","Type":"ContainerStarted","Data":"d564d11c8f0df8c2766979276a32644bc0b66492324716d3966919c1ca369016"}
Sep 14 08:19:51.583932 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:51.583870    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn" podStartSLOduration=1.583847994 podStartE2EDuration="1.583847994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:19:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:19:51.580495113 +0000 UTC m=+1846.285033616" watchObservedRunningTime="2026-09-14 08:19:51.583847994 +0000 UTC m=+1846.288386497"
Sep 14 08:19:52.186756 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:52.186719    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:52.191730 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:52.191705    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:52.566924 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:52.566843    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:52.567803 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:52.567781    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:19:52.828887 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:52.828810    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:53.464655 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:53.464621    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:53.668173 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:53.668144    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:53.765535 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:53.765457    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:54.177180 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:54.177149    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:54.827619 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:54.827591    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:55.677966 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:55.677939    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:55.778151 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:55.778120    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:55.902864 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:55.902835    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:56.264862 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:56.264809    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:56.347101 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:56.347048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:56.666121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:56.666091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:57.428776 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:19:57.428745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:00.306479 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:00.306447    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:20:02.032240 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:02.032180    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:02.125446 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:02.125410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:02.807748 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:02.807715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:08.608424 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:08.608381    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 14 08:20:08.608915 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:08.608455    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn" podUID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 14 08:20:10.668691 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:10.668662    2576 generic.go:356] "Generic (PLEG): container finished" podID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerID="85c60f8ea2561006f1a9e14c7071b380d9e7003bf1dab70d79743af53d969c3e" exitCode=0
Sep 14 08:20:10.669025 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:10.668716    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn" event={"ID":"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de","Type":"ContainerDied","Data":"85c60f8ea2561006f1a9e14c7071b380d9e7003bf1dab70d79743af53d969c3e"}
Sep 14 08:20:10.669089 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:10.669076    2576 scope.go:117] "RemoveContainer" containerID="85c60f8ea2561006f1a9e14c7071b380d9e7003bf1dab70d79743af53d969c3e"
Sep 14 08:20:10.670254 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:10.670202    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:20:10.670323 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:10.670298    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:20:10.670358 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:10.670336    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:20:11.675442 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:11.675403    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn" event={"ID":"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de","Type":"ContainerStarted","Data":"67e93675c49dbfa0e4f737e811b678ca2c6f1efaeab7a01b90594044dafdf982"}
Sep 14 08:20:12.187008 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:12.186975    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:20:12.187176 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:12.187148    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 14 08:20:12.187243 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:12.187225    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn" podUID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 14 08:20:13.186705 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:13.186666    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 14 08:20:13.187098 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:13.186729    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn" podUID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 14 08:20:14.186629 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:14.186590    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 14 08:20:14.186835 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:14.186656    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn" podUID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 14 08:20:15.186198 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:15.186162    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 14 08:20:15.186394 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:15.186234    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn" podUID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 14 08:20:16.186323 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:16.186285    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 14 08:20:16.186728 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:16.186355    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn" podUID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 14 08:20:16.657028 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:16.656997    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:17.345424 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:17.345390    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:20:17.345909 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:17.345885    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:20:17.346624 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:17.346602    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:20:18.040143 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:18.040110    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:18.577552 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:18.577525    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:18.857652 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:18.857626    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:19.488062 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:19.488032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:20.251850 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:20.251805    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:20.889236 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:20.889183    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:30.674316 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:30.674235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:32.716243 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:32.716169    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:36.692847 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:36.692809    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:20:39.611837 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:39.611803    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:39.664231 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:39.664176    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:58.814985 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:20:58.814951    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:21:05.654417 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:05.654382    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:05.944764 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:05.944684    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"]
Sep 14 08:21:05.947547 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:05.947516    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:05.949915 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:05.949892    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-70893f-openshift-default-dockercfg-h76dx\""
Sep 14 08:21:05.962686 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:05.962658    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"]
Sep 14 08:21:05.971535 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:05.971510    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:06.067326 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.067266    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-credential-socket\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.067524 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.067351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-podinfo\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.067524 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.067449    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-token\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.067524 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.067477    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-data\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.067524 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.067500    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istiod-ca-cert\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.067761 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.067542    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-envoy\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.067761 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.067584    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qqtn\" (UniqueName: \"kubernetes.io/projected/aaad0931-b20f-4991-ba91-d6a9bfacbdad-kube-api-access-5qqtn\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.067761 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.067611    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-workload-certs\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.067761 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.067658    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-workload-socket\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.085678 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.085651    2576 scope.go:117] "RemoveContainer" containerID="c2d32ba4f8287445a82645f0ca28b1d56970c3d5fd52d18ce5e09fd11bb15980"
Sep 14 08:21:06.094387 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.094363    2576 scope.go:117] "RemoveContainer" containerID="8ec50cad890ae4888723ad2bb3f52dcc4d65ededa210939c6d673cb0b3601e0d"
Sep 14 08:21:06.102449 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.102428    2576 scope.go:117] "RemoveContainer" containerID="18a3d1a9933060c0f156362fce4650a79e28083103fbb2b9efa10f9ae3545782"
Sep 14 08:21:06.135136 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.135090    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:06.168408 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.168387    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-podinfo\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.168507 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.168448    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-token\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.168507 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.168475    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-data\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.168507 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.168490    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istiod-ca-cert\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.168662 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.168520    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-envoy\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.169437 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.168555    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqtn\" (UniqueName: \"kubernetes.io/projected/aaad0931-b20f-4991-ba91-d6a9bfacbdad-kube-api-access-5qqtn\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.169437 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.169105    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-workload-certs\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.169437 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.169197    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-workload-socket\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.169437 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.169315    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-credential-socket\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.169699 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.169600    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-workload-certs\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.169699 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.169661    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-workload-socket\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.176867 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.169908    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istiod-ca-cert\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.176867 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.170012    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-credential-socket\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.176867 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.170028    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-data\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.176867 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.172049    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-podinfo\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.177378 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.177355    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-envoy\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.177687 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.177664    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-token\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.178090 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.178066    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qqtn\" (UniqueName: \"kubernetes.io/projected/aaad0931-b20f-4991-ba91-d6a9bfacbdad-kube-api-access-5qqtn\") pod \"e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") " pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.257640 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.257549    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:06.538410 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.538380    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"]
Sep 14 08:21:06.540375 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:21:06.540347    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaad0931_b20f_4991_ba91_d6a9bfacbdad.slice/crio-29f7db3e247242f9d27eefb6bb11d9ec9479d73c1a27de93d9334ff945d7e76f WatchSource:0}: Error finding container 29f7db3e247242f9d27eefb6bb11d9ec9479d73c1a27de93d9334ff945d7e76f: Status 404 returned error can't find the container with id 29f7db3e247242f9d27eefb6bb11d9ec9479d73c1a27de93d9334ff945d7e76f
Sep 14 08:21:06.542556 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.542517    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:21:06.542657 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.542584    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:21:06.542657 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.542612    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:21:06.968610 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.968574    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49" event={"ID":"aaad0931-b20f-4991-ba91-d6a9bfacbdad","Type":"ContainerStarted","Data":"769337e35833ce5545aa97455df1057cef9640c7e7d779f482428a4e5e02f027"}
Sep 14 08:21:06.968974 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.968626    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49" event={"ID":"aaad0931-b20f-4991-ba91-d6a9bfacbdad","Type":"ContainerStarted","Data":"29f7db3e247242f9d27eefb6bb11d9ec9479d73c1a27de93d9334ff945d7e76f"}
Sep 14 08:21:06.990520 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:06.990470    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49" podStartSLOduration=1.990453897 podStartE2EDuration="1.990453897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:21:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:21:06.987489501 +0000 UTC m=+1921.692028004" watchObservedRunningTime="2026-09-14 08:21:06.990453897 +0000 UTC m=+1921.694992399"
Sep 14 08:21:07.258596 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:07.258500    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:07.263993 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:07.263960    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:07.888960 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:07.888920    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:07.973111 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:07.973076    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:07.973946 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:07.973928    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:07.989136 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:07.989088    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:08.342159 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:08.342082    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:08.935292 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:08.935253    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:09.032756 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:09.032729    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:09.133516 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:09.133477    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:09.849259 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:09.849201    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:10.717999 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:10.717966    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:11.437459 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:11.437422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:12.082104 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:12.082073    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:12.133897 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:12.133867    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:12.457481 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:12.457450    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:12.560727 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:12.560697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:12.637171 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:12.637113    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:17.896062 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:17.896018    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:21:28.082226 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:28.082180    2576 generic.go:356] "Generic (PLEG): container finished" podID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerID="769337e35833ce5545aa97455df1057cef9640c7e7d779f482428a4e5e02f027" exitCode=0
Sep 14 08:21:28.082652 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:28.082239    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49" event={"ID":"aaad0931-b20f-4991-ba91-d6a9bfacbdad","Type":"ContainerDied","Data":"769337e35833ce5545aa97455df1057cef9640c7e7d779f482428a4e5e02f027"}
Sep 14 08:21:28.082696 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:28.082668    2576 scope.go:117] "RemoveContainer" containerID="769337e35833ce5545aa97455df1057cef9640c7e7d779f482428a4e5e02f027"
Sep 14 08:21:28.083882 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:28.083848    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:21:28.083953 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:28.083921    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:21:28.083990 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:28.083955    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 14 08:21:29.089297 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:29.089265    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49" event={"ID":"aaad0931-b20f-4991-ba91-d6a9bfacbdad","Type":"ContainerStarted","Data":"a4b74807d8da181cc6bf015e3ec1d4c70b4e6dc3137cc03f5a1abf5942497118"}
Sep 14 08:21:29.257757 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:29.257721    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:29.257921 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:29.257897    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 14 08:21:29.257970 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:29.257945    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49" podUID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 14 08:21:30.258847 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:30.258807    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 14 08:21:30.259294 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:30.258863    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49" podUID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 14 08:21:31.258685 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:31.258641    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 14 08:21:31.258870 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:31.258702    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49" podUID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 14 08:21:32.258796 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:32.258760    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 14 08:21:32.258980 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:32.258816    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49" podUID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 14 08:21:33.257986 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:33.257938    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 14 08:21:33.258195 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:33.258010    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49" podUID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 14 08:21:33.833603 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:33.833562    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:33.963854 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:33.963820    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:34.262904 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:34.262871    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:34.263279 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:34.263241    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:34.264744 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:34.264723    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:34.473288 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:34.473253    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:35.351556 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:35.351519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:35.981580 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:35.981542    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:36.636014 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:36.635977    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:37.351964 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:37.351926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:38.005231 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:38.005175    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:38.634594 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:38.634562    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:40.351496 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:40.351432    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:40.895260 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:40.895197    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:40.955951 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:40.955910    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:41.055020 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:41.054988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:41.147401 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:41.147313    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:41.450803 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:41.450711    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:41.533307 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:41.533277    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:42.139319 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:42.139284    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:42.995671 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:42.995641    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:43.735973 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:43.735941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:44.353755 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:44.353725    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:45.590029 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:45.589995    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:21:47.305568 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:47.305514    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:47.381401 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:47.381354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:47.555735 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:47.555659    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"]
Sep 14 08:21:47.555939 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:47.555912    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49" podUID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerName="istio-proxy" containerID="cri-o://a4b74807d8da181cc6bf015e3ec1d4c70b4e6dc3137cc03f5a1abf5942497118" gracePeriod=30
Sep 14 08:21:47.572578 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:47.572554    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:49.266067 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:49.266026    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 08:21:49.266452 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:49.266092    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49" podUID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:21:51.648513 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:51.648479    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:52.420168 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.420137    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:52.807305 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.807281    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:52.846393 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.846368    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istiod-ca-cert\") pod \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") "
Sep 14 08:21:52.846528 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.846414    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-envoy\") pod \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") "
Sep 14 08:21:52.846528 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.846455    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-podinfo\") pod \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") "
Sep 14 08:21:52.846528 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.846496    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-workload-socket\") pod \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") "
Sep 14 08:21:52.846714 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.846539    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qqtn\" (UniqueName: \"kubernetes.io/projected/aaad0931-b20f-4991-ba91-d6a9bfacbdad-kube-api-access-5qqtn\") pod \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") "
Sep 14 08:21:52.846714 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.846575    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-workload-certs\") pod \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") "
Sep 14 08:21:52.846714 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.846614    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-data\") pod \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") "
Sep 14 08:21:52.846714 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.846682    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-token\") pod \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") "
Sep 14 08:21:52.846714 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.846705    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-credential-socket\") pod \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\" (UID: \"aaad0931-b20f-4991-ba91-d6a9bfacbdad\") "
Sep 14 08:21:52.846970 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.846795    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aaad0931-b20f-4991-ba91-d6a9bfacbdad" (UID: "aaad0931-b20f-4991-ba91-d6a9bfacbdad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:21:52.846970 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.846942    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aaad0931-b20f-4991-ba91-d6a9bfacbdad" (UID: "aaad0931-b20f-4991-ba91-d6a9bfacbdad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:21:52.847452 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.847094    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aaad0931-b20f-4991-ba91-d6a9bfacbdad" (UID: "aaad0931-b20f-4991-ba91-d6a9bfacbdad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:21:52.847568 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.847169    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-data" (OuterVolumeSpecName: "istio-data") pod "aaad0931-b20f-4991-ba91-d6a9bfacbdad" (UID: "aaad0931-b20f-4991-ba91-d6a9bfacbdad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:21:52.847568 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.847157    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:21:52.847568 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.847547    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:21:52.847568 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.847185    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aaad0931-b20f-4991-ba91-d6a9bfacbdad" (UID: "aaad0931-b20f-4991-ba91-d6a9bfacbdad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:21:52.849055 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.849026    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaad0931-b20f-4991-ba91-d6a9bfacbdad-kube-api-access-5qqtn" (OuterVolumeSpecName: "kube-api-access-5qqtn") pod "aaad0931-b20f-4991-ba91-d6a9bfacbdad" (UID: "aaad0931-b20f-4991-ba91-d6a9bfacbdad"). InnerVolumeSpecName "kube-api-access-5qqtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:21:52.849346 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.849318    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-token" (OuterVolumeSpecName: "istio-token") pod "aaad0931-b20f-4991-ba91-d6a9bfacbdad" (UID: "aaad0931-b20f-4991-ba91-d6a9bfacbdad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:21:52.849503 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.849485    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aaad0931-b20f-4991-ba91-d6a9bfacbdad" (UID: "aaad0931-b20f-4991-ba91-d6a9bfacbdad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:21:52.849564 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.849488    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aaad0931-b20f-4991-ba91-d6a9bfacbdad" (UID: "aaad0931-b20f-4991-ba91-d6a9bfacbdad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:21:52.948022 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.947945    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:21:52.948022 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.947980    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:21:52.948022 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.947991    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:21:52.948022 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.947999    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:21:52.948022 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.948006    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:21:52.948022 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.948015    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qqtn\" (UniqueName: \"kubernetes.io/projected/aaad0931-b20f-4991-ba91-d6a9bfacbdad-kube-api-access-5qqtn\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:21:52.948022 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:52.948023    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaad0931-b20f-4991-ba91-d6a9bfacbdad-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:21:53.127776 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.127740    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:53.236059 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.231294    2576 generic.go:356] "Generic (PLEG): container finished" podID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerID="a4b74807d8da181cc6bf015e3ec1d4c70b4e6dc3137cc03f5a1abf5942497118" exitCode=0
Sep 14 08:21:53.236059 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.231454    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49" event={"ID":"aaad0931-b20f-4991-ba91-d6a9bfacbdad","Type":"ContainerDied","Data":"a4b74807d8da181cc6bf015e3ec1d4c70b4e6dc3137cc03f5a1abf5942497118"}
Sep 14 08:21:53.236059 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.231541    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49" event={"ID":"aaad0931-b20f-4991-ba91-d6a9bfacbdad","Type":"ContainerDied","Data":"29f7db3e247242f9d27eefb6bb11d9ec9479d73c1a27de93d9334ff945d7e76f"}
Sep 14 08:21:53.236059 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.231589    2576 scope.go:117] "RemoveContainer" containerID="a4b74807d8da181cc6bf015e3ec1d4c70b4e6dc3137cc03f5a1abf5942497118"
Sep 14 08:21:53.236059 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.231813    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"
Sep 14 08:21:53.247032 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.247012    2576 scope.go:117] "RemoveContainer" containerID="769337e35833ce5545aa97455df1057cef9640c7e7d779f482428a4e5e02f027"
Sep 14 08:21:53.255115 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.255081    2576 scope.go:117] "RemoveContainer" containerID="a4b74807d8da181cc6bf015e3ec1d4c70b4e6dc3137cc03f5a1abf5942497118"
Sep 14 08:21:53.255839 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:21:53.255416    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4b74807d8da181cc6bf015e3ec1d4c70b4e6dc3137cc03f5a1abf5942497118\": container with ID starting with a4b74807d8da181cc6bf015e3ec1d4c70b4e6dc3137cc03f5a1abf5942497118 not found: ID does not exist" containerID="a4b74807d8da181cc6bf015e3ec1d4c70b4e6dc3137cc03f5a1abf5942497118"
Sep 14 08:21:53.255839 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.255465    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4b74807d8da181cc6bf015e3ec1d4c70b4e6dc3137cc03f5a1abf5942497118"} err="failed to get container status \"a4b74807d8da181cc6bf015e3ec1d4c70b4e6dc3137cc03f5a1abf5942497118\": rpc error: code = NotFound desc = could not find container \"a4b74807d8da181cc6bf015e3ec1d4c70b4e6dc3137cc03f5a1abf5942497118\": container with ID starting with a4b74807d8da181cc6bf015e3ec1d4c70b4e6dc3137cc03f5a1abf5942497118 not found: ID does not exist"
Sep 14 08:21:53.255839 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.255485    2576 scope.go:117] "RemoveContainer" containerID="769337e35833ce5545aa97455df1057cef9640c7e7d779f482428a4e5e02f027"
Sep 14 08:21:53.255839 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:21:53.255808    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"769337e35833ce5545aa97455df1057cef9640c7e7d779f482428a4e5e02f027\": container with ID starting with 769337e35833ce5545aa97455df1057cef9640c7e7d779f482428a4e5e02f027 not found: ID does not exist" containerID="769337e35833ce5545aa97455df1057cef9640c7e7d779f482428a4e5e02f027"
Sep 14 08:21:53.256121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.255865    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"769337e35833ce5545aa97455df1057cef9640c7e7d779f482428a4e5e02f027"} err="failed to get container status \"769337e35833ce5545aa97455df1057cef9640c7e7d779f482428a4e5e02f027\": rpc error: code = NotFound desc = could not find container \"769337e35833ce5545aa97455df1057cef9640c7e7d779f482428a4e5e02f027\": container with ID starting with 769337e35833ce5545aa97455df1057cef9640c7e7d779f482428a4e5e02f027 not found: ID does not exist"
Sep 14 08:21:53.258186 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.258165    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"]
Sep 14 08:21:53.263492 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.263466    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-70893f-openshift-default-869875566b-kbz49"]
Sep 14 08:21:53.752183 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.752150    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:53.839007 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:53.838966    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" path="/var/lib/kubelet/pods/aaad0931-b20f-4991-ba91-d6a9bfacbdad/volumes"
Sep 14 08:21:54.632046 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:54.632016    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:54.958018 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:54.957930    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:55.262235 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:21:55.262139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:04.694710 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:04.694674    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:22:06.131264 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:06.131230    2576 scope.go:117] "RemoveContainer" containerID="3c4b12f0ad1bd5dd5378e75f13a9047f160160e9c28dfdbd53bef4eea928abb0"
Sep 14 08:22:06.910718 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:06.910684    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"]
Sep 14 08:22:06.910976 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:06.910930    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z" podUID="a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65" containerName="maas-api" containerID="cri-o://746beb3854d2dba7f566eb7a365dc7ba4f97955acf2d0f9434ac84fc4e586134" gracePeriod=30
Sep 14 08:22:06.942196 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:06.942170    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:07.042376 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.042350    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:07.119565 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.119535    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:07.159074 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.159045    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:22:07.184746 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.184674    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:07.279143 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.279113    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-maas-api-tls\") pod \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") "
Sep 14 08:22:07.279326 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.279203    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-metrics-certs\") pod \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") "
Sep 14 08:22:07.279326 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.279288    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-openshift-service-ca\") pod \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") "
Sep 14 08:22:07.279401 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.279364    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2qm8\" (UniqueName: \"kubernetes.io/projected/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-kube-api-access-p2qm8\") pod \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\" (UID: \"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65\") "
Sep 14 08:22:07.279637 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.279610    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65" (UID: "a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:22:07.281523 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.281488    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65" (UID: "a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:22:07.281523 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.281503    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-kube-api-access-p2qm8" (OuterVolumeSpecName: "kube-api-access-p2qm8") pod "a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65" (UID: "a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65"). InnerVolumeSpecName "kube-api-access-p2qm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:22:07.281668 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.281522    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65" (UID: "a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:22:07.317736 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.317677    2576 generic.go:356] "Generic (PLEG): container finished" podID="a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65" containerID="746beb3854d2dba7f566eb7a365dc7ba4f97955acf2d0f9434ac84fc4e586134" exitCode=0
Sep 14 08:22:07.317934 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.317720    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z" event={"ID":"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65","Type":"ContainerDied","Data":"746beb3854d2dba7f566eb7a365dc7ba4f97955acf2d0f9434ac84fc4e586134"}
Sep 14 08:22:07.317934 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.317761    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"
Sep 14 08:22:07.317934 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.317813    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z" event={"ID":"a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65","Type":"ContainerDied","Data":"1b2b1c19723d90878ab061ac8413a6248235a8c195ef30ae36f7478a4e721229"}
Sep 14 08:22:07.317934 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.317845    2576 scope.go:117] "RemoveContainer" containerID="746beb3854d2dba7f566eb7a365dc7ba4f97955acf2d0f9434ac84fc4e586134"
Sep 14 08:22:07.331153 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.331130    2576 scope.go:117] "RemoveContainer" containerID="746beb3854d2dba7f566eb7a365dc7ba4f97955acf2d0f9434ac84fc4e586134"
Sep 14 08:22:07.331526 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:22:07.331497    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"746beb3854d2dba7f566eb7a365dc7ba4f97955acf2d0f9434ac84fc4e586134\": container with ID starting with 746beb3854d2dba7f566eb7a365dc7ba4f97955acf2d0f9434ac84fc4e586134 not found: ID does not exist" containerID="746beb3854d2dba7f566eb7a365dc7ba4f97955acf2d0f9434ac84fc4e586134"
Sep 14 08:22:07.331585 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.331537    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"746beb3854d2dba7f566eb7a365dc7ba4f97955acf2d0f9434ac84fc4e586134"} err="failed to get container status \"746beb3854d2dba7f566eb7a365dc7ba4f97955acf2d0f9434ac84fc4e586134\": rpc error: code = NotFound desc = could not find container \"746beb3854d2dba7f566eb7a365dc7ba4f97955acf2d0f9434ac84fc4e586134\": container with ID starting with 746beb3854d2dba7f566eb7a365dc7ba4f97955acf2d0f9434ac84fc4e586134 not found: ID does not exist"
Sep 14 08:22:07.344046 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.344018    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"]
Sep 14 08:22:07.347909 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.347885    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-03d6fa-75cf665875-pc29z"]
Sep 14 08:22:07.381021 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.380996    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-metrics-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:22:07.381021 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.381019    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-openshift-service-ca\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:22:07.381167 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.381029    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2qm8\" (UniqueName: \"kubernetes.io/projected/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-kube-api-access-p2qm8\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:22:07.381167 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.381038    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65-maas-api-tls\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:22:07.512776 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.512700    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:07.612648 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.612619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:07.838173 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:07.838095    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65" path="/var/lib/kubelet/pods/a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65/volumes"
Sep 14 08:22:08.228984 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:08.228948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:08.865627 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:08.865595    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:09.639541 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:09.639507    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:12.538141 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:12.538101    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:16.689735 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:16.689698    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:22:48.958473 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:48.958437    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:49.752974 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:49.752940    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:55.169881 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:22:55.169840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:09.132051 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:09.132014    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:09.198876 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:09.198836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:09.396962 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:09.396884    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:09.939353 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:09.939323    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:10.832117 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:10.832084    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:11.526684 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:11.526651    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:12.169606 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:12.169571    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:12.824024 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:12.823993    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:20.231546 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:20.231502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:20.372680 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:20.372615    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:20.461448 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:20.461421    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:20.529953 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:20.529886    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:20.585924 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:20.585898    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:20.911109 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:20.911078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:21.010403 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:21.010375    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:21.566234 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:21.566186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:22.317091 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:22.317059    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:23.297268 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:23.297229    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:23:26.497119 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:26.497084    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:26.563518 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:26.563486    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:26.748661 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:26.748579    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"]
Sep 14 08:23:26.748871 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:26.748836    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn" podUID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerName="istio-proxy" containerID="cri-o://67e93675c49dbfa0e4f737e811b678ca2c6f1efaeab7a01b90594044dafdf982" gracePeriod=30
Sep 14 08:23:26.768069 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:26.768046    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:27.302234 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:27.302183    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:27.963611 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:27.963579    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:28.745981 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:28.745945    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:29.408526 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:29.408493    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:31.984860 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:31.984823    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:23:32.063883 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.063820    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-credential-socket\") pod \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") "
Sep 14 08:23:32.064002 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.063883    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-workload-certs\") pod \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") "
Sep 14 08:23:32.064002 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.063919    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-data\") pod \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") "
Sep 14 08:23:32.064002 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.063947    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istiod-ca-cert\") pod \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") "
Sep 14 08:23:32.064002 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.063997    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-envoy\") pod \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") "
Sep 14 08:23:32.064242 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.064024    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2k259\" (UniqueName: \"kubernetes.io/projected/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-kube-api-access-2k259\") pod \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") "
Sep 14 08:23:32.064242 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.064058    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-token\") pod \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") "
Sep 14 08:23:32.064242 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.064099    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-workload-socket\") pod \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") "
Sep 14 08:23:32.064242 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.064102    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" (UID: "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:32.064466 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.064239    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-podinfo\") pod \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\" (UID: \"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de\") "
Sep 14 08:23:32.064466 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.064240    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" (UID: "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:32.064466 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.064249    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-data" (OuterVolumeSpecName: "istio-data") pod "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" (UID: "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:32.064466 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.064356    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" (UID: "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:32.064466 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.064435    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" (UID: "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:32.064706 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.064632    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:32.064706 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.064652    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:32.064706 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.064666    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:32.064706 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.064679    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:32.064706 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.064691    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:32.066269 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.066195    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" (UID: "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:23:32.066392 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.066303    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-token" (OuterVolumeSpecName: "istio-token") pod "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" (UID: "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:32.066637 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.066611    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-kube-api-access-2k259" (OuterVolumeSpecName: "kube-api-access-2k259") pod "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" (UID: "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de"). InnerVolumeSpecName "kube-api-access-2k259". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:32.066637 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.066618    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" (UID: "fbdc18a7-9bef-4859-8017-6b9b3c3ad5de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:32.165955 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.165932    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:32.165955 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.165955    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2k259\" (UniqueName: \"kubernetes.io/projected/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-kube-api-access-2k259\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:32.166082 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.165965    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:32.166082 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.165974    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:32.753414 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.753380    2576 generic.go:356] "Generic (PLEG): container finished" podID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerID="67e93675c49dbfa0e4f737e811b678ca2c6f1efaeab7a01b90594044dafdf982" exitCode=0
Sep 14 08:23:32.753593 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.753440    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"
Sep 14 08:23:32.753593 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.753452    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn" event={"ID":"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de","Type":"ContainerDied","Data":"67e93675c49dbfa0e4f737e811b678ca2c6f1efaeab7a01b90594044dafdf982"}
Sep 14 08:23:32.753593 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.753493    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn" event={"ID":"fbdc18a7-9bef-4859-8017-6b9b3c3ad5de","Type":"ContainerDied","Data":"d564d11c8f0df8c2766979276a32644bc0b66492324716d3966919c1ca369016"}
Sep 14 08:23:32.753593 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.753514    2576 scope.go:117] "RemoveContainer" containerID="67e93675c49dbfa0e4f737e811b678ca2c6f1efaeab7a01b90594044dafdf982"
Sep 14 08:23:32.762813 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.762795    2576 scope.go:117] "RemoveContainer" containerID="85c60f8ea2561006f1a9e14c7071b380d9e7003bf1dab70d79743af53d969c3e"
Sep 14 08:23:32.770739 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.770720    2576 scope.go:117] "RemoveContainer" containerID="67e93675c49dbfa0e4f737e811b678ca2c6f1efaeab7a01b90594044dafdf982"
Sep 14 08:23:32.771022 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:23:32.771003    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67e93675c49dbfa0e4f737e811b678ca2c6f1efaeab7a01b90594044dafdf982\": container with ID starting with 67e93675c49dbfa0e4f737e811b678ca2c6f1efaeab7a01b90594044dafdf982 not found: ID does not exist" containerID="67e93675c49dbfa0e4f737e811b678ca2c6f1efaeab7a01b90594044dafdf982"
Sep 14 08:23:32.771068 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.771030    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67e93675c49dbfa0e4f737e811b678ca2c6f1efaeab7a01b90594044dafdf982"} err="failed to get container status \"67e93675c49dbfa0e4f737e811b678ca2c6f1efaeab7a01b90594044dafdf982\": rpc error: code = NotFound desc = could not find container \"67e93675c49dbfa0e4f737e811b678ca2c6f1efaeab7a01b90594044dafdf982\": container with ID starting with 67e93675c49dbfa0e4f737e811b678ca2c6f1efaeab7a01b90594044dafdf982 not found: ID does not exist"
Sep 14 08:23:32.771068 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.771047    2576 scope.go:117] "RemoveContainer" containerID="85c60f8ea2561006f1a9e14c7071b380d9e7003bf1dab70d79743af53d969c3e"
Sep 14 08:23:32.771339 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:23:32.771317    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85c60f8ea2561006f1a9e14c7071b380d9e7003bf1dab70d79743af53d969c3e\": container with ID starting with 85c60f8ea2561006f1a9e14c7071b380d9e7003bf1dab70d79743af53d969c3e not found: ID does not exist" containerID="85c60f8ea2561006f1a9e14c7071b380d9e7003bf1dab70d79743af53d969c3e"
Sep 14 08:23:32.771433 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.771343    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85c60f8ea2561006f1a9e14c7071b380d9e7003bf1dab70d79743af53d969c3e"} err="failed to get container status \"85c60f8ea2561006f1a9e14c7071b380d9e7003bf1dab70d79743af53d969c3e\": rpc error: code = NotFound desc = could not find container \"85c60f8ea2561006f1a9e14c7071b380d9e7003bf1dab70d79743af53d969c3e\": container with ID starting with 85c60f8ea2561006f1a9e14c7071b380d9e7003bf1dab70d79743af53d969c3e not found: ID does not exist"
Sep 14 08:23:32.777971 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.777949    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"]
Sep 14 08:23:32.781896 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:32.781877    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-7bcf7b-openshift-default-55c9f74db-dvrbn"]
Sep 14 08:23:33.837865 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:33.837824    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" path="/var/lib/kubelet/pods/fbdc18a7-9bef-4859-8017-6b9b3c3ad5de/volumes"
Sep 14 08:23:37.743745 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:37.743709    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"]
Sep 14 08:23:37.744148 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:37.743943    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v" podUID="05230cb1-58cf-4530-b293-3019188d3361" containerName="maas-api" containerID="cri-o://e533d03502337102b4538d87f8f59dce2c91b18b302a6ed8dbc7dad46d1fa1ac" gracePeriod=30
Sep 14 08:23:37.862360 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:37.862335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:37.963583 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:37.963555    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:37.990117 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:37.990094    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:23:38.064477 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.064403    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:38.120665 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.120614    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-metrics-certs\") pod \"05230cb1-58cf-4530-b293-3019188d3361\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") "
Sep 14 08:23:38.120859 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.120716    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbrfx\" (UniqueName: \"kubernetes.io/projected/05230cb1-58cf-4530-b293-3019188d3361-kube-api-access-nbrfx\") pod \"05230cb1-58cf-4530-b293-3019188d3361\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") "
Sep 14 08:23:38.120859 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.120754    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-maas-api-tls\") pod \"05230cb1-58cf-4530-b293-3019188d3361\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") "
Sep 14 08:23:38.120859 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.120786    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05230cb1-58cf-4530-b293-3019188d3361-openshift-service-ca\") pod \"05230cb1-58cf-4530-b293-3019188d3361\" (UID: \"05230cb1-58cf-4530-b293-3019188d3361\") "
Sep 14 08:23:38.121441 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.121412    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05230cb1-58cf-4530-b293-3019188d3361-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "05230cb1-58cf-4530-b293-3019188d3361" (UID: "05230cb1-58cf-4530-b293-3019188d3361"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:38.124667 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.124547    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "05230cb1-58cf-4530-b293-3019188d3361" (UID: "05230cb1-58cf-4530-b293-3019188d3361"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:23:38.124667 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.124631    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05230cb1-58cf-4530-b293-3019188d3361-kube-api-access-nbrfx" (OuterVolumeSpecName: "kube-api-access-nbrfx") pod "05230cb1-58cf-4530-b293-3019188d3361" (UID: "05230cb1-58cf-4530-b293-3019188d3361"). InnerVolumeSpecName "kube-api-access-nbrfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:38.127243 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.124889    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "05230cb1-58cf-4530-b293-3019188d3361" (UID: "05230cb1-58cf-4530-b293-3019188d3361"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:23:38.221671 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.221633    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-metrics-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:38.221671 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.221664    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbrfx\" (UniqueName: \"kubernetes.io/projected/05230cb1-58cf-4530-b293-3019188d3361-kube-api-access-nbrfx\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:38.221671 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.221676    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05230cb1-58cf-4530-b293-3019188d3361-maas-api-tls\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:38.221892 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.221685    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05230cb1-58cf-4530-b293-3019188d3361-openshift-service-ca\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:38.365647 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.365619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:38.557773 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.557743    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:38.787172 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.787088    2576 generic.go:356] "Generic (PLEG): container finished" podID="05230cb1-58cf-4530-b293-3019188d3361" containerID="e533d03502337102b4538d87f8f59dce2c91b18b302a6ed8dbc7dad46d1fa1ac" exitCode=0
Sep 14 08:23:38.787172 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.787135    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"
Sep 14 08:23:38.787685 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.787182    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v" event={"ID":"05230cb1-58cf-4530-b293-3019188d3361","Type":"ContainerDied","Data":"e533d03502337102b4538d87f8f59dce2c91b18b302a6ed8dbc7dad46d1fa1ac"}
Sep 14 08:23:38.787685 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.787262    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v" event={"ID":"05230cb1-58cf-4530-b293-3019188d3361","Type":"ContainerDied","Data":"774b7160a79d8a57739b180392d25874c89cfc1e291773e565bab431be84e866"}
Sep 14 08:23:38.787685 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.787299    2576 scope.go:117] "RemoveContainer" containerID="e533d03502337102b4538d87f8f59dce2c91b18b302a6ed8dbc7dad46d1fa1ac"
Sep 14 08:23:38.797142 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.797126    2576 scope.go:117] "RemoveContainer" containerID="e533d03502337102b4538d87f8f59dce2c91b18b302a6ed8dbc7dad46d1fa1ac"
Sep 14 08:23:38.797440 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:23:38.797419    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e533d03502337102b4538d87f8f59dce2c91b18b302a6ed8dbc7dad46d1fa1ac\": container with ID starting with e533d03502337102b4538d87f8f59dce2c91b18b302a6ed8dbc7dad46d1fa1ac not found: ID does not exist" containerID="e533d03502337102b4538d87f8f59dce2c91b18b302a6ed8dbc7dad46d1fa1ac"
Sep 14 08:23:38.797519 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.797452    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e533d03502337102b4538d87f8f59dce2c91b18b302a6ed8dbc7dad46d1fa1ac"} err="failed to get container status \"e533d03502337102b4538d87f8f59dce2c91b18b302a6ed8dbc7dad46d1fa1ac\": rpc error: code = NotFound desc = could not find container \"e533d03502337102b4538d87f8f59dce2c91b18b302a6ed8dbc7dad46d1fa1ac\": container with ID starting with e533d03502337102b4538d87f8f59dce2c91b18b302a6ed8dbc7dad46d1fa1ac not found: ID does not exist"
Sep 14 08:23:38.808623 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.808600    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"]
Sep 14 08:23:38.812721 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:38.812696    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-919cc9-7fd4997d6-tvl8v"]
Sep 14 08:23:39.106134 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:39.106103    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:39.838862 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:39.838830    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05230cb1-58cf-4530-b293-3019188d3361" path="/var/lib/kubelet/pods/05230cb1-58cf-4530-b293-3019188d3361/volumes"
Sep 14 08:23:40.500291 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:40.500257    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:23:44.270024 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:44.269990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:44.373606 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:44.373579    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:44.528538 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:44.528454    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"]
Sep 14 08:23:44.528708 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:44.528654    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq" podUID="31b495c9-37d7-4774-8335-9b3e04049f64" containerName="istio-proxy" containerID="cri-o://f6d74d1940aa920efb3cbaf4ab915a6ca01b1cd0ca036555922dd429fa3b0743" gracePeriod=30
Sep 14 08:23:44.553172 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:44.553121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:45.742845 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:45.742792    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:46.366908 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:46.366856    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:49.781106 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.781080    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:23:49.846647 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.846555    2576 generic.go:356] "Generic (PLEG): container finished" podID="31b495c9-37d7-4774-8335-9b3e04049f64" containerID="f6d74d1940aa920efb3cbaf4ab915a6ca01b1cd0ca036555922dd429fa3b0743" exitCode=0
Sep 14 08:23:49.846647 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.846595    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"
Sep 14 08:23:49.846836 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.846643    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq" event={"ID":"31b495c9-37d7-4774-8335-9b3e04049f64","Type":"ContainerDied","Data":"f6d74d1940aa920efb3cbaf4ab915a6ca01b1cd0ca036555922dd429fa3b0743"}
Sep 14 08:23:49.846836 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.846691    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq" event={"ID":"31b495c9-37d7-4774-8335-9b3e04049f64","Type":"ContainerDied","Data":"43e9a6a8bfd5f7665653779123a483e14b23399d18e5f5ca613d5489cabf1eee"}
Sep 14 08:23:49.846836 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.846713    2576 scope.go:117] "RemoveContainer" containerID="f6d74d1940aa920efb3cbaf4ab915a6ca01b1cd0ca036555922dd429fa3b0743"
Sep 14 08:23:49.855433 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.855416    2576 scope.go:117] "RemoveContainer" containerID="04bbf7fa6beb0d61c53b2afcdfa16905d6dd59a5579ce014dcd9c279876d7722"
Sep 14 08:23:49.863044 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.863025    2576 scope.go:117] "RemoveContainer" containerID="f6d74d1940aa920efb3cbaf4ab915a6ca01b1cd0ca036555922dd429fa3b0743"
Sep 14 08:23:49.863403 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:23:49.863379    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6d74d1940aa920efb3cbaf4ab915a6ca01b1cd0ca036555922dd429fa3b0743\": container with ID starting with f6d74d1940aa920efb3cbaf4ab915a6ca01b1cd0ca036555922dd429fa3b0743 not found: ID does not exist" containerID="f6d74d1940aa920efb3cbaf4ab915a6ca01b1cd0ca036555922dd429fa3b0743"
Sep 14 08:23:49.863495 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.863410    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6d74d1940aa920efb3cbaf4ab915a6ca01b1cd0ca036555922dd429fa3b0743"} err="failed to get container status \"f6d74d1940aa920efb3cbaf4ab915a6ca01b1cd0ca036555922dd429fa3b0743\": rpc error: code = NotFound desc = could not find container \"f6d74d1940aa920efb3cbaf4ab915a6ca01b1cd0ca036555922dd429fa3b0743\": container with ID starting with f6d74d1940aa920efb3cbaf4ab915a6ca01b1cd0ca036555922dd429fa3b0743 not found: ID does not exist"
Sep 14 08:23:49.863495 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.863435    2576 scope.go:117] "RemoveContainer" containerID="04bbf7fa6beb0d61c53b2afcdfa16905d6dd59a5579ce014dcd9c279876d7722"
Sep 14 08:23:49.863696 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:23:49.863676    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04bbf7fa6beb0d61c53b2afcdfa16905d6dd59a5579ce014dcd9c279876d7722\": container with ID starting with 04bbf7fa6beb0d61c53b2afcdfa16905d6dd59a5579ce014dcd9c279876d7722 not found: ID does not exist" containerID="04bbf7fa6beb0d61c53b2afcdfa16905d6dd59a5579ce014dcd9c279876d7722"
Sep 14 08:23:49.863733 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.863705    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04bbf7fa6beb0d61c53b2afcdfa16905d6dd59a5579ce014dcd9c279876d7722"} err="failed to get container status \"04bbf7fa6beb0d61c53b2afcdfa16905d6dd59a5579ce014dcd9c279876d7722\": rpc error: code = NotFound desc = could not find container \"04bbf7fa6beb0d61c53b2afcdfa16905d6dd59a5579ce014dcd9c279876d7722\": container with ID starting with 04bbf7fa6beb0d61c53b2afcdfa16905d6dd59a5579ce014dcd9c279876d7722 not found: ID does not exist"
Sep 14 08:23:49.930691 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.930654    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-workload-certs\") pod \"31b495c9-37d7-4774-8335-9b3e04049f64\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") "
Sep 14 08:23:49.930691 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.930699    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-workload-socket\") pod \"31b495c9-37d7-4774-8335-9b3e04049f64\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") "
Sep 14 08:23:49.930943 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.930752    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31b495c9-37d7-4774-8335-9b3e04049f64-istiod-ca-cert\") pod \"31b495c9-37d7-4774-8335-9b3e04049f64\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") "
Sep 14 08:23:49.930943 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.930769    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-istio-data\") pod \"31b495c9-37d7-4774-8335-9b3e04049f64\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") "
Sep 14 08:23:49.930943 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.930799    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31b495c9-37d7-4774-8335-9b3e04049f64-istio-podinfo\") pod \"31b495c9-37d7-4774-8335-9b3e04049f64\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") "
Sep 14 08:23:49.930943 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.930823    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-istio-envoy\") pod \"31b495c9-37d7-4774-8335-9b3e04049f64\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") "
Sep 14 08:23:49.930943 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.930873    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31b495c9-37d7-4774-8335-9b3e04049f64-istio-token\") pod \"31b495c9-37d7-4774-8335-9b3e04049f64\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") "
Sep 14 08:23:49.930943 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.930914    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9l4k\" (UniqueName: \"kubernetes.io/projected/31b495c9-37d7-4774-8335-9b3e04049f64-kube-api-access-r9l4k\") pod \"31b495c9-37d7-4774-8335-9b3e04049f64\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") "
Sep 14 08:23:49.931196 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.930963    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-credential-socket\") pod \"31b495c9-37d7-4774-8335-9b3e04049f64\" (UID: \"31b495c9-37d7-4774-8335-9b3e04049f64\") "
Sep 14 08:23:49.931196 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.931010    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "31b495c9-37d7-4774-8335-9b3e04049f64" (UID: "31b495c9-37d7-4774-8335-9b3e04049f64"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:49.931196 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.931100    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "31b495c9-37d7-4774-8335-9b3e04049f64" (UID: "31b495c9-37d7-4774-8335-9b3e04049f64"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:49.931196 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.931166    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-istio-data" (OuterVolumeSpecName: "istio-data") pod "31b495c9-37d7-4774-8335-9b3e04049f64" (UID: "31b495c9-37d7-4774-8335-9b3e04049f64"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:49.931397 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.931265    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31b495c9-37d7-4774-8335-9b3e04049f64-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "31b495c9-37d7-4774-8335-9b3e04049f64" (UID: "31b495c9-37d7-4774-8335-9b3e04049f64"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:49.931445 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.931402    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "31b495c9-37d7-4774-8335-9b3e04049f64" (UID: "31b495c9-37d7-4774-8335-9b3e04049f64"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:49.931523 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.931502    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:49.931669 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.931648    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:49.931731 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.931672    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31b495c9-37d7-4774-8335-9b3e04049f64-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:49.931731 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.931688    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:49.933495 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.933459    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31b495c9-37d7-4774-8335-9b3e04049f64-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "31b495c9-37d7-4774-8335-9b3e04049f64" (UID: "31b495c9-37d7-4774-8335-9b3e04049f64"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:23:49.933609 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.933523    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31b495c9-37d7-4774-8335-9b3e04049f64-kube-api-access-r9l4k" (OuterVolumeSpecName: "kube-api-access-r9l4k") pod "31b495c9-37d7-4774-8335-9b3e04049f64" (UID: "31b495c9-37d7-4774-8335-9b3e04049f64"). InnerVolumeSpecName "kube-api-access-r9l4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:49.933609 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.933590    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "31b495c9-37d7-4774-8335-9b3e04049f64" (UID: "31b495c9-37d7-4774-8335-9b3e04049f64"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:49.933726 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:49.933676    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31b495c9-37d7-4774-8335-9b3e04049f64-istio-token" (OuterVolumeSpecName: "istio-token") pod "31b495c9-37d7-4774-8335-9b3e04049f64" (UID: "31b495c9-37d7-4774-8335-9b3e04049f64"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:50.032968 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:50.032929    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9l4k\" (UniqueName: \"kubernetes.io/projected/31b495c9-37d7-4774-8335-9b3e04049f64-kube-api-access-r9l4k\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:50.032968 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:50.032961    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:50.032968 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:50.032974    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31b495c9-37d7-4774-8335-9b3e04049f64-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:50.033249 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:50.032987    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31b495c9-37d7-4774-8335-9b3e04049f64-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:50.033249 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:50.032998    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31b495c9-37d7-4774-8335-9b3e04049f64-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:50.169352 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:50.169310    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"]
Sep 14 08:23:50.172933 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:50.172904    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-919cc9-openshift-default-5bc995dd8-nhphq"]
Sep 14 08:23:51.838296 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:51.838263    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31b495c9-37d7-4774-8335-9b3e04049f64" path="/var/lib/kubelet/pods/31b495c9-37d7-4774-8335-9b3e04049f64/volumes"
Sep 14 08:23:55.392791 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.392755    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"]
Sep 14 08:23:55.393287 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.392962    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm" podUID="af3c165c-c4e4-421f-a51d-244bfc0e39fb" containerName="maas-api" containerID="cri-o://db96b9f4c2e16710423eea664e2b23b578ab944ae64a016eeff1e3f867d1dd51" gracePeriod=30
Sep 14 08:23:55.423781 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.423752    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:55.489898 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.489867    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:55.552740 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.552708    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:55.619000 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.618821    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:55.636262 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.636240    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:23:55.685572 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.685503    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af3c165c-c4e4-421f-a51d-244bfc0e39fb-openshift-service-ca\") pod \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") "
Sep 14 08:23:55.685572 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.685537    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-maas-api-tls\") pod \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") "
Sep 14 08:23:55.685572 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.685568    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-622qg\" (UniqueName: \"kubernetes.io/projected/af3c165c-c4e4-421f-a51d-244bfc0e39fb-kube-api-access-622qg\") pod \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") "
Sep 14 08:23:55.685804 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.685642    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-metrics-certs\") pod \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\" (UID: \"af3c165c-c4e4-421f-a51d-244bfc0e39fb\") "
Sep 14 08:23:55.685934 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.685906    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af3c165c-c4e4-421f-a51d-244bfc0e39fb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "af3c165c-c4e4-421f-a51d-244bfc0e39fb" (UID: "af3c165c-c4e4-421f-a51d-244bfc0e39fb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:55.687831 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.687802    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af3c165c-c4e4-421f-a51d-244bfc0e39fb-kube-api-access-622qg" (OuterVolumeSpecName: "kube-api-access-622qg") pod "af3c165c-c4e4-421f-a51d-244bfc0e39fb" (UID: "af3c165c-c4e4-421f-a51d-244bfc0e39fb"). InnerVolumeSpecName "kube-api-access-622qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:55.687831 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.687819    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "af3c165c-c4e4-421f-a51d-244bfc0e39fb" (UID: "af3c165c-c4e4-421f-a51d-244bfc0e39fb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:23:55.687994 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.687876    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "af3c165c-c4e4-421f-a51d-244bfc0e39fb" (UID: "af3c165c-c4e4-421f-a51d-244bfc0e39fb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:23:55.787315 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.787277    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-622qg\" (UniqueName: \"kubernetes.io/projected/af3c165c-c4e4-421f-a51d-244bfc0e39fb-kube-api-access-622qg\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:55.787315 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.787311    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-metrics-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:55.787315 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.787321    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af3c165c-c4e4-421f-a51d-244bfc0e39fb-openshift-service-ca\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:55.787618 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.787329    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af3c165c-c4e4-421f-a51d-244bfc0e39fb-maas-api-tls\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:23:55.876939 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.876906    2576 generic.go:356] "Generic (PLEG): container finished" podID="af3c165c-c4e4-421f-a51d-244bfc0e39fb" containerID="db96b9f4c2e16710423eea664e2b23b578ab944ae64a016eeff1e3f867d1dd51" exitCode=0
Sep 14 08:23:55.877074 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.876950    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"
Sep 14 08:23:55.877074 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.876962    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm" event={"ID":"af3c165c-c4e4-421f-a51d-244bfc0e39fb","Type":"ContainerDied","Data":"db96b9f4c2e16710423eea664e2b23b578ab944ae64a016eeff1e3f867d1dd51"}
Sep 14 08:23:55.877074 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.877007    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm" event={"ID":"af3c165c-c4e4-421f-a51d-244bfc0e39fb","Type":"ContainerDied","Data":"5c5793252bd8e7d2d18de0f56ff26b37e50c18090ee961e9750a745b00cf3762"}
Sep 14 08:23:55.877074 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.877037    2576 scope.go:117] "RemoveContainer" containerID="db96b9f4c2e16710423eea664e2b23b578ab944ae64a016eeff1e3f867d1dd51"
Sep 14 08:23:55.886042 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.886014    2576 scope.go:117] "RemoveContainer" containerID="db96b9f4c2e16710423eea664e2b23b578ab944ae64a016eeff1e3f867d1dd51"
Sep 14 08:23:55.886327 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:23:55.886307    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db96b9f4c2e16710423eea664e2b23b578ab944ae64a016eeff1e3f867d1dd51\": container with ID starting with db96b9f4c2e16710423eea664e2b23b578ab944ae64a016eeff1e3f867d1dd51 not found: ID does not exist" containerID="db96b9f4c2e16710423eea664e2b23b578ab944ae64a016eeff1e3f867d1dd51"
Sep 14 08:23:55.886381 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.886336    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db96b9f4c2e16710423eea664e2b23b578ab944ae64a016eeff1e3f867d1dd51"} err="failed to get container status \"db96b9f4c2e16710423eea664e2b23b578ab944ae64a016eeff1e3f867d1dd51\": rpc error: code = NotFound desc = could not find container \"db96b9f4c2e16710423eea664e2b23b578ab944ae64a016eeff1e3f867d1dd51\": container with ID starting with db96b9f4c2e16710423eea664e2b23b578ab944ae64a016eeff1e3f867d1dd51 not found: ID does not exist"
Sep 14 08:23:55.892012 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.891991    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"]
Sep 14 08:23:55.895743 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:55.895721    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc0d6e-66f6db797-kfzkm"]
Sep 14 08:23:56.005263 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:56.005148    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:56.105556 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:56.105520    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:57.842324 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:57.842288    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af3c165c-c4e4-421f-a51d-244bfc0e39fb" path="/var/lib/kubelet/pods/af3c165c-c4e4-421f-a51d-244bfc0e39fb/volumes"
Sep 14 08:23:58.100385 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:23:58.100300    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:24:02.031350 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:02.031321    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:24:02.131667 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:02.131604    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:24:02.283357 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:02.283274    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"]
Sep 14 08:24:02.283573 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:02.283547    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7" podUID="df5656ab-c018-4e50-8f58-005290d96ee9" containerName="istio-proxy" containerID="cri-o://30ac2cd8c4349b20b96b4cd1bb24f90396b5f3b84d9bcd67360ed01dac7fcf32" gracePeriod=30
Sep 14 08:24:02.676534 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:02.676481    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:24:06.995673 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:06.995639    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:24:07.540420 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.540397    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:24:07.594220 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594146    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df5656ab-c018-4e50-8f58-005290d96ee9-istio-token\") pod \"df5656ab-c018-4e50-8f58-005290d96ee9\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") "
Sep 14 08:24:07.594220 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594183    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-credential-socket\") pod \"df5656ab-c018-4e50-8f58-005290d96ee9\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") "
Sep 14 08:24:07.594395 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594233    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df5656ab-c018-4e50-8f58-005290d96ee9-istio-podinfo\") pod \"df5656ab-c018-4e50-8f58-005290d96ee9\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") "
Sep 14 08:24:07.594395 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594257    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df5656ab-c018-4e50-8f58-005290d96ee9-istiod-ca-cert\") pod \"df5656ab-c018-4e50-8f58-005290d96ee9\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") "
Sep 14 08:24:07.594395 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594286    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jk8rx\" (UniqueName: \"kubernetes.io/projected/df5656ab-c018-4e50-8f58-005290d96ee9-kube-api-access-jk8rx\") pod \"df5656ab-c018-4e50-8f58-005290d96ee9\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") "
Sep 14 08:24:07.594395 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594329    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-workload-socket\") pod \"df5656ab-c018-4e50-8f58-005290d96ee9\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") "
Sep 14 08:24:07.594395 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594373    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-workload-certs\") pod \"df5656ab-c018-4e50-8f58-005290d96ee9\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") "
Sep 14 08:24:07.594651 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594402    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-istio-envoy\") pod \"df5656ab-c018-4e50-8f58-005290d96ee9\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") "
Sep 14 08:24:07.594651 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594466    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-istio-data\") pod \"df5656ab-c018-4e50-8f58-005290d96ee9\" (UID: \"df5656ab-c018-4e50-8f58-005290d96ee9\") "
Sep 14 08:24:07.594651 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594480    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "df5656ab-c018-4e50-8f58-005290d96ee9" (UID: "df5656ab-c018-4e50-8f58-005290d96ee9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:24:07.594814 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594677    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "df5656ab-c018-4e50-8f58-005290d96ee9" (UID: "df5656ab-c018-4e50-8f58-005290d96ee9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:24:07.594814 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594728    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df5656ab-c018-4e50-8f58-005290d96ee9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "df5656ab-c018-4e50-8f58-005290d96ee9" (UID: "df5656ab-c018-4e50-8f58-005290d96ee9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:24:07.594814 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594741    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-credential-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:24:07.594970 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594856    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "df5656ab-c018-4e50-8f58-005290d96ee9" (UID: "df5656ab-c018-4e50-8f58-005290d96ee9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:24:07.594970 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.594783    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-workload-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:24:07.595080 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.595052    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-istio-data" (OuterVolumeSpecName: "istio-data") pod "df5656ab-c018-4e50-8f58-005290d96ee9" (UID: "df5656ab-c018-4e50-8f58-005290d96ee9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:24:07.596677 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.596649    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "df5656ab-c018-4e50-8f58-005290d96ee9" (UID: "df5656ab-c018-4e50-8f58-005290d96ee9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:24:07.596808 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.596653    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df5656ab-c018-4e50-8f58-005290d96ee9-kube-api-access-jk8rx" (OuterVolumeSpecName: "kube-api-access-jk8rx") pod "df5656ab-c018-4e50-8f58-005290d96ee9" (UID: "df5656ab-c018-4e50-8f58-005290d96ee9"). InnerVolumeSpecName "kube-api-access-jk8rx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:24:07.596881 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.596815    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df5656ab-c018-4e50-8f58-005290d96ee9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "df5656ab-c018-4e50-8f58-005290d96ee9" (UID: "df5656ab-c018-4e50-8f58-005290d96ee9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:24:07.596881 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.596842    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df5656ab-c018-4e50-8f58-005290d96ee9-istio-token" (OuterVolumeSpecName: "istio-token") pod "df5656ab-c018-4e50-8f58-005290d96ee9" (UID: "df5656ab-c018-4e50-8f58-005290d96ee9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:24:07.695566 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.695542    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df5656ab-c018-4e50-8f58-005290d96ee9-istiod-ca-cert\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:24:07.695675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.695566    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk8rx\" (UniqueName: \"kubernetes.io/projected/df5656ab-c018-4e50-8f58-005290d96ee9-kube-api-access-jk8rx\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:24:07.695675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.695581    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-workload-socket\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:24:07.695675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.695593    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-istio-envoy\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:24:07.695675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.695606    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df5656ab-c018-4e50-8f58-005290d96ee9-istio-data\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:24:07.695675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.695618    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df5656ab-c018-4e50-8f58-005290d96ee9-istio-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:24:07.695675 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.695629    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df5656ab-c018-4e50-8f58-005290d96ee9-istio-podinfo\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:24:07.939119 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.939082    2576 generic.go:356] "Generic (PLEG): container finished" podID="df5656ab-c018-4e50-8f58-005290d96ee9" containerID="30ac2cd8c4349b20b96b4cd1bb24f90396b5f3b84d9bcd67360ed01dac7fcf32" exitCode=0
Sep 14 08:24:07.939286 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.939155    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"
Sep 14 08:24:07.939286 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.939165    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7" event={"ID":"df5656ab-c018-4e50-8f58-005290d96ee9","Type":"ContainerDied","Data":"30ac2cd8c4349b20b96b4cd1bb24f90396b5f3b84d9bcd67360ed01dac7fcf32"}
Sep 14 08:24:07.939286 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.939247    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7" event={"ID":"df5656ab-c018-4e50-8f58-005290d96ee9","Type":"ContainerDied","Data":"9b54065960f872a066b9c67111d5be6d3215f70e72361a652f8beec40b968c2f"}
Sep 14 08:24:07.939286 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.939282    2576 scope.go:117] "RemoveContainer" containerID="30ac2cd8c4349b20b96b4cd1bb24f90396b5f3b84d9bcd67360ed01dac7fcf32"
Sep 14 08:24:07.950123 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.950100    2576 scope.go:117] "RemoveContainer" containerID="378d8d5d485f2ba82626f009903103d130752d5625d073a70476f28ae89a1a4e"
Sep 14 08:24:07.956470 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.956444    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"]
Sep 14 08:24:07.958034 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.958015    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fc0d6e-openshift-default-dc4fd8454-fnsk7"]
Sep 14 08:24:07.960942 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.960921    2576 scope.go:117] "RemoveContainer" containerID="30ac2cd8c4349b20b96b4cd1bb24f90396b5f3b84d9bcd67360ed01dac7fcf32"
Sep 14 08:24:07.961237 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:24:07.961202    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30ac2cd8c4349b20b96b4cd1bb24f90396b5f3b84d9bcd67360ed01dac7fcf32\": container with ID starting with 30ac2cd8c4349b20b96b4cd1bb24f90396b5f3b84d9bcd67360ed01dac7fcf32 not found: ID does not exist" containerID="30ac2cd8c4349b20b96b4cd1bb24f90396b5f3b84d9bcd67360ed01dac7fcf32"
Sep 14 08:24:07.961337 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.961244    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ac2cd8c4349b20b96b4cd1bb24f90396b5f3b84d9bcd67360ed01dac7fcf32"} err="failed to get container status \"30ac2cd8c4349b20b96b4cd1bb24f90396b5f3b84d9bcd67360ed01dac7fcf32\": rpc error: code = NotFound desc = could not find container \"30ac2cd8c4349b20b96b4cd1bb24f90396b5f3b84d9bcd67360ed01dac7fcf32\": container with ID starting with 30ac2cd8c4349b20b96b4cd1bb24f90396b5f3b84d9bcd67360ed01dac7fcf32 not found: ID does not exist"
Sep 14 08:24:07.961337 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.961259    2576 scope.go:117] "RemoveContainer" containerID="378d8d5d485f2ba82626f009903103d130752d5625d073a70476f28ae89a1a4e"
Sep 14 08:24:07.961541 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:24:07.961523    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"378d8d5d485f2ba82626f009903103d130752d5625d073a70476f28ae89a1a4e\": container with ID starting with 378d8d5d485f2ba82626f009903103d130752d5625d073a70476f28ae89a1a4e not found: ID does not exist" containerID="378d8d5d485f2ba82626f009903103d130752d5625d073a70476f28ae89a1a4e"
Sep 14 08:24:07.961610 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:07.961544    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"378d8d5d485f2ba82626f009903103d130752d5625d073a70476f28ae89a1a4e"} err="failed to get container status \"378d8d5d485f2ba82626f009903103d130752d5625d073a70476f28ae89a1a4e\": rpc error: code = NotFound desc = could not find container \"378d8d5d485f2ba82626f009903103d130752d5625d073a70476f28ae89a1a4e\": container with ID starting with 378d8d5d485f2ba82626f009903103d130752d5625d073a70476f28ae89a1a4e not found: ID does not exist"
Sep 14 08:24:09.838189 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:09.838160    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df5656ab-c018-4e50-8f58-005290d96ee9" path="/var/lib/kubelet/pods/df5656ab-c018-4e50-8f58-005290d96ee9/volumes"
Sep 14 08:24:43.107359 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:43.107323    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:24:43.160089 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:24:43.160060    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:25:26.837280 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:25:26.837240    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:25:27.396076 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:25:27.396040    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:25:28.756092 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:25:28.756052    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:25:53.890508 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:25:53.890472    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:26:01.601400 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:26:01.601354    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:26:10.092868 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:26:10.092835    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:26:17.693424 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:26:17.693387    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:26:17.778586 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:26:17.778555    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:26:26.697854 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:26:26.697815    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:26:32.695129 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:26:32.695094    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:27:41.070831 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:27:41.070796    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:27:41.148959 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:27:41.148927    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:27:59.894717 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:27:59.894678    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:28:08.405164 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:08.405123    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:28:13.677951 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.677914    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"]
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678279    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31b495c9-37d7-4774-8335-9b3e04049f64" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678291    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="31b495c9-37d7-4774-8335-9b3e04049f64" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678305    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af3c165c-c4e4-421f-a51d-244bfc0e39fb" containerName="maas-api"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678310    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="af3c165c-c4e4-421f-a51d-244bfc0e39fb" containerName="maas-api"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678318    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678323    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678334    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df5656ab-c018-4e50-8f58-005290d96ee9" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678339    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="df5656ab-c018-4e50-8f58-005290d96ee9" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678345    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678350    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678356    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678362    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678367    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678372    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678382    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df5656ab-c018-4e50-8f58-005290d96ee9" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678388    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="df5656ab-c018-4e50-8f58-005290d96ee9" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678395    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65" containerName="maas-api"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678400    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65" containerName="maas-api"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678406    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05230cb1-58cf-4530-b293-3019188d3361" containerName="maas-api"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678412    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="05230cb1-58cf-4530-b293-3019188d3361" containerName="maas-api"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678474    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerName="istio-proxy"
Sep 14 08:28:13.678469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678481    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="05230cb1-58cf-4530-b293-3019188d3361" containerName="maas-api"
Sep 14 08:28:13.679139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678488    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="31b495c9-37d7-4774-8335-9b3e04049f64" containerName="istio-proxy"
Sep 14 08:28:13.679139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678493    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbdc18a7-9bef-4859-8017-6b9b3c3ad5de" containerName="istio-proxy"
Sep 14 08:28:13.679139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678500    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1f1dc8f-1aa7-4136-bb03-6a82f30a3b65" containerName="maas-api"
Sep 14 08:28:13.679139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678505    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="31b495c9-37d7-4774-8335-9b3e04049f64" containerName="istio-proxy"
Sep 14 08:28:13.679139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678511    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="af3c165c-c4e4-421f-a51d-244bfc0e39fb" containerName="maas-api"
Sep 14 08:28:13.679139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678518    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerName="istio-proxy"
Sep 14 08:28:13.679139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678523    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaad0931-b20f-4991-ba91-d6a9bfacbdad" containerName="istio-proxy"
Sep 14 08:28:13.679139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678528    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="df5656ab-c018-4e50-8f58-005290d96ee9" containerName="istio-proxy"
Sep 14 08:28:13.679139 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.678533    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="df5656ab-c018-4e50-8f58-005290d96ee9" containerName="istio-proxy"
Sep 14 08:28:13.681414 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.681392    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.683754 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.683730    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-7066a4c2-kserve-self-signed-certs\""
Sep 14 08:28:13.689469 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.689446    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"]
Sep 14 08:28:13.782274 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.782237    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-home\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.782460 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.782299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnrq4\" (UniqueName: \"kubernetes.io/projected/93717a8e-c672-4137-9181-faa3a669fd46-kube-api-access-cnrq4\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.782460 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.782342    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-tmp-dir\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.782460 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.782418    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-dshm\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.782460 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.782459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93717a8e-c672-4137-9181-faa3a669fd46-tls-certs\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.782642 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.782491    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-model-cache\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.785085 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.785060    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:28:13.882953 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.882913    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-tmp-dir\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.883108 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.882963    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-dshm\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.883108 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.883083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93717a8e-c672-4137-9181-faa3a669fd46-tls-certs\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.883202 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.883150    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-model-cache\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.883283 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.883236    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-home\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.883327 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.883300    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnrq4\" (UniqueName: \"kubernetes.io/projected/93717a8e-c672-4137-9181-faa3a669fd46-kube-api-access-cnrq4\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.883489 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.883471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-tmp-dir\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.883557 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.883514    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-model-cache\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.883608 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.883541    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-home\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.885484 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.885457    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-dshm\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.885706 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.885692    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93717a8e-c672-4137-9181-faa3a669fd46-tls-certs\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.891067 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.891044    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnrq4\" (UniqueName: \"kubernetes.io/projected/93717a8e-c672-4137-9181-faa3a669fd46-kube-api-access-cnrq4\") pod \"e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") " pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:13.991073 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:13.990978    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:14.265179 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:14.265152    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"]
Sep 14 08:28:14.267138 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:28:14.267112    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93717a8e_c672_4137_9181_faa3a669fd46.slice/crio-688bc225e9ce53ef8100142a898b5a0793f2570f9fce7efd115fe410a17e75d3 WatchSource:0}: Error finding container 688bc225e9ce53ef8100142a898b5a0793f2570f9fce7efd115fe410a17e75d3: Status 404 returned error can't find the container with id 688bc225e9ce53ef8100142a898b5a0793f2570f9fce7efd115fe410a17e75d3
Sep 14 08:28:15.076501 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:15.076468    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld" event={"ID":"93717a8e-c672-4137-9181-faa3a669fd46","Type":"ContainerStarted","Data":"4ba72a8525cb8458bdbb2c1d1f7887814b5d223b77745d89be4186903ac5bf49"}
Sep 14 08:28:15.076879 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:15.076508    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld" event={"ID":"93717a8e-c672-4137-9181-faa3a669fd46","Type":"ContainerStarted","Data":"688bc225e9ce53ef8100142a898b5a0793f2570f9fce7efd115fe410a17e75d3"}
Sep 14 08:28:15.093606 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:15.093552    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld" podStartSLOduration=2.093535143 podStartE2EDuration="2.093535143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:28:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:28:15.092289162 +0000 UTC m=+2349.796827667" watchObservedRunningTime="2026-09-14 08:28:15.093535143 +0000 UTC m=+2349.798073645"
Sep 14 08:28:15.790157 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:15.790121    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:28:23.992093 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:23.992055    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:23.992093 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:23.992102    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:24.004283 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:24.004258    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:24.129899 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:24.129876    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:28:31.384961 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:31.384931    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:28:34.190944 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:34.190912    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:28:53.856968 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:53.856931    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:28:53.957719 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:28:53.957687    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:29:03.606877 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:03.606841    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:29:03.657603 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:03.657578    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:29:04.836367 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:04.836327    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"]
Sep 14 08:29:04.854640 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:04.854606    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:29:06.407172 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:06.407144    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:29:06.902288 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:06.902251    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:29:15.396431 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:15.396396    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:29:19.856115 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:19.856076    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld" podUID="93717a8e-c672-4137-9181-faa3a669fd46" containerName="main" containerID="cri-o://4ba72a8525cb8458bdbb2c1d1f7887814b5d223b77745d89be4186903ac5bf49" gracePeriod=45
Sep 14 08:29:20.103200 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.103175    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:29:20.140609 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.140552    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93717a8e-c672-4137-9181-faa3a669fd46-tls-certs\") pod \"93717a8e-c672-4137-9181-faa3a669fd46\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") "
Sep 14 08:29:20.140718 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.140622    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-dshm\") pod \"93717a8e-c672-4137-9181-faa3a669fd46\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") "
Sep 14 08:29:20.140718 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.140648    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-home\") pod \"93717a8e-c672-4137-9181-faa3a669fd46\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") "
Sep 14 08:29:20.140718 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.140696    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-tmp-dir\") pod \"93717a8e-c672-4137-9181-faa3a669fd46\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") "
Sep 14 08:29:20.140885 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.140728    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-model-cache\") pod \"93717a8e-c672-4137-9181-faa3a669fd46\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") "
Sep 14 08:29:20.140885 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.140783    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnrq4\" (UniqueName: \"kubernetes.io/projected/93717a8e-c672-4137-9181-faa3a669fd46-kube-api-access-cnrq4\") pod \"93717a8e-c672-4137-9181-faa3a669fd46\" (UID: \"93717a8e-c672-4137-9181-faa3a669fd46\") "
Sep 14 08:29:20.140993 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.140932    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-home" (OuterVolumeSpecName: "home") pod "93717a8e-c672-4137-9181-faa3a669fd46" (UID: "93717a8e-c672-4137-9181-faa3a669fd46"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:29:20.141047 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.140999    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "93717a8e-c672-4137-9181-faa3a669fd46" (UID: "93717a8e-c672-4137-9181-faa3a669fd46"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:29:20.141091 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.141015    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-model-cache" (OuterVolumeSpecName: "model-cache") pod "93717a8e-c672-4137-9181-faa3a669fd46" (UID: "93717a8e-c672-4137-9181-faa3a669fd46"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:29:20.141200 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.141182    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-tmp-dir\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:29:20.141297 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.141219    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-model-cache\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:29:20.141297 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.141233    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-home\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:29:20.142776 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.142754    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-dshm" (OuterVolumeSpecName: "dshm") pod "93717a8e-c672-4137-9181-faa3a669fd46" (UID: "93717a8e-c672-4137-9181-faa3a669fd46"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:29:20.142870 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.142839    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93717a8e-c672-4137-9181-faa3a669fd46-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "93717a8e-c672-4137-9181-faa3a669fd46" (UID: "93717a8e-c672-4137-9181-faa3a669fd46"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:29:20.142870 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.142856    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93717a8e-c672-4137-9181-faa3a669fd46-kube-api-access-cnrq4" (OuterVolumeSpecName: "kube-api-access-cnrq4") pod "93717a8e-c672-4137-9181-faa3a669fd46" (UID: "93717a8e-c672-4137-9181-faa3a669fd46"). InnerVolumeSpecName "kube-api-access-cnrq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:29:20.242618 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.242586    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnrq4\" (UniqueName: \"kubernetes.io/projected/93717a8e-c672-4137-9181-faa3a669fd46-kube-api-access-cnrq4\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:29:20.242618 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.242612    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93717a8e-c672-4137-9181-faa3a669fd46-tls-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:29:20.242618 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.242621    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93717a8e-c672-4137-9181-faa3a669fd46-dshm\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:29:20.382723 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.382692    2576 generic.go:356] "Generic (PLEG): container finished" podID="93717a8e-c672-4137-9181-faa3a669fd46" containerID="4ba72a8525cb8458bdbb2c1d1f7887814b5d223b77745d89be4186903ac5bf49" exitCode=0
Sep 14 08:29:20.382873 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.382764    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"
Sep 14 08:29:20.382873 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.382785    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld" event={"ID":"93717a8e-c672-4137-9181-faa3a669fd46","Type":"ContainerDied","Data":"4ba72a8525cb8458bdbb2c1d1f7887814b5d223b77745d89be4186903ac5bf49"}
Sep 14 08:29:20.382873 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.382832    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld" event={"ID":"93717a8e-c672-4137-9181-faa3a669fd46","Type":"ContainerDied","Data":"688bc225e9ce53ef8100142a898b5a0793f2570f9fce7efd115fe410a17e75d3"}
Sep 14 08:29:20.382873 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.382851    2576 scope.go:117] "RemoveContainer" containerID="4ba72a8525cb8458bdbb2c1d1f7887814b5d223b77745d89be4186903ac5bf49"
Sep 14 08:29:20.391719 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.391699    2576 scope.go:117] "RemoveContainer" containerID="4ba72a8525cb8458bdbb2c1d1f7887814b5d223b77745d89be4186903ac5bf49"
Sep 14 08:29:20.392031 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:29:20.392000    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ba72a8525cb8458bdbb2c1d1f7887814b5d223b77745d89be4186903ac5bf49\": container with ID starting with 4ba72a8525cb8458bdbb2c1d1f7887814b5d223b77745d89be4186903ac5bf49 not found: ID does not exist" containerID="4ba72a8525cb8458bdbb2c1d1f7887814b5d223b77745d89be4186903ac5bf49"
Sep 14 08:29:20.392132 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.392040    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ba72a8525cb8458bdbb2c1d1f7887814b5d223b77745d89be4186903ac5bf49"} err="failed to get container status \"4ba72a8525cb8458bdbb2c1d1f7887814b5d223b77745d89be4186903ac5bf49\": rpc error: code = NotFound desc = could not find container \"4ba72a8525cb8458bdbb2c1d1f7887814b5d223b77745d89be4186903ac5bf49\": container with ID starting with 4ba72a8525cb8458bdbb2c1d1f7887814b5d223b77745d89be4186903ac5bf49 not found: ID does not exist"
Sep 14 08:29:20.403720 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.403692    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"]
Sep 14 08:29:20.407121 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:20.407096    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-7066a4c2-kserve-5cfd9c4bc-xjzld"]
Sep 14 08:29:21.795055 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:21.795011    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:29:21.838835 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:21.838801    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93717a8e-c672-4137-9181-faa3a669fd46" path="/var/lib/kubelet/pods/93717a8e-c672-4137-9181-faa3a669fd46/volumes"
Sep 14 08:29:26.637302 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.637261    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"]
Sep 14 08:29:26.637894 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.637792    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31b495c9-37d7-4774-8335-9b3e04049f64" containerName="istio-proxy"
Sep 14 08:29:26.637894 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.637811    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="31b495c9-37d7-4774-8335-9b3e04049f64" containerName="istio-proxy"
Sep 14 08:29:26.637894 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.637830    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93717a8e-c672-4137-9181-faa3a669fd46" containerName="main"
Sep 14 08:29:26.637894 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.637839    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="93717a8e-c672-4137-9181-faa3a669fd46" containerName="main"
Sep 14 08:29:26.638112 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.637942    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="93717a8e-c672-4137-9181-faa3a669fd46" containerName="main"
Sep 14 08:29:26.642586 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.642564    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:26.645836 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.645805    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:29:26.645978 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.645869    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 08:29:26.645978 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.645878    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 08:29:26.645978 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.645955    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zwz44\""
Sep 14 08:29:26.645978 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.645971    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 14 08:29:26.650621 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.650599    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"]
Sep 14 08:29:26.652827 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.652803    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"]
Sep 14 08:29:26.656070 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.656053    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:26.666757 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.666731    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"]
Sep 14 08:29:26.694899 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.694877    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-maas-api-tls\") pod \"maas-api-7f5c8b48b7-bv67p\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:26.695001 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.694905    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df730895-1ff7-47dc-b459-514ac4ec0c16-metrics-certs\") pod \"maas-api-579fddb5f-jt9h2\" (UID: \"df730895-1ff7-47dc-b459-514ac4ec0c16\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:26.695001 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.694928    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-openshift-service-ca\") pod \"maas-api-7f5c8b48b7-bv67p\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:26.695001 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.694953    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df730895-1ff7-47dc-b459-514ac4ec0c16-openshift-service-ca\") pod \"maas-api-579fddb5f-jt9h2\" (UID: \"df730895-1ff7-47dc-b459-514ac4ec0c16\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:26.695001 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.694977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-metrics-certs\") pod \"maas-api-7f5c8b48b7-bv67p\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:26.695001 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.694993    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrd2k\" (UniqueName: \"kubernetes.io/projected/df730895-1ff7-47dc-b459-514ac4ec0c16-kube-api-access-wrd2k\") pod \"maas-api-579fddb5f-jt9h2\" (UID: \"df730895-1ff7-47dc-b459-514ac4ec0c16\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:26.695173 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.695017    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df730895-1ff7-47dc-b459-514ac4ec0c16-maas-api-tls\") pod \"maas-api-579fddb5f-jt9h2\" (UID: \"df730895-1ff7-47dc-b459-514ac4ec0c16\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:26.695173 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.695064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bgx6\" (UniqueName: \"kubernetes.io/projected/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-kube-api-access-8bgx6\") pod \"maas-api-7f5c8b48b7-bv67p\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:26.795432 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.795407    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-openshift-service-ca\") pod \"maas-api-7f5c8b48b7-bv67p\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:26.795545 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.795441    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df730895-1ff7-47dc-b459-514ac4ec0c16-openshift-service-ca\") pod \"maas-api-579fddb5f-jt9h2\" (UID: \"df730895-1ff7-47dc-b459-514ac4ec0c16\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:26.795545 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.795458    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-metrics-certs\") pod \"maas-api-7f5c8b48b7-bv67p\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:26.795545 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.795481    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrd2k\" (UniqueName: \"kubernetes.io/projected/df730895-1ff7-47dc-b459-514ac4ec0c16-kube-api-access-wrd2k\") pod \"maas-api-579fddb5f-jt9h2\" (UID: \"df730895-1ff7-47dc-b459-514ac4ec0c16\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:26.795545 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.795517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df730895-1ff7-47dc-b459-514ac4ec0c16-maas-api-tls\") pod \"maas-api-579fddb5f-jt9h2\" (UID: \"df730895-1ff7-47dc-b459-514ac4ec0c16\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:26.795766 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.795573    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bgx6\" (UniqueName: \"kubernetes.io/projected/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-kube-api-access-8bgx6\") pod \"maas-api-7f5c8b48b7-bv67p\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:26.795766 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.795634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-maas-api-tls\") pod \"maas-api-7f5c8b48b7-bv67p\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:26.795766 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.795658    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df730895-1ff7-47dc-b459-514ac4ec0c16-metrics-certs\") pod \"maas-api-579fddb5f-jt9h2\" (UID: \"df730895-1ff7-47dc-b459-514ac4ec0c16\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:26.796196 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.796178    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df730895-1ff7-47dc-b459-514ac4ec0c16-openshift-service-ca\") pod \"maas-api-579fddb5f-jt9h2\" (UID: \"df730895-1ff7-47dc-b459-514ac4ec0c16\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:26.796328 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.796230    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-openshift-service-ca\") pod \"maas-api-7f5c8b48b7-bv67p\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:26.798001 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.797977    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df730895-1ff7-47dc-b459-514ac4ec0c16-metrics-certs\") pod \"maas-api-579fddb5f-jt9h2\" (UID: \"df730895-1ff7-47dc-b459-514ac4ec0c16\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:26.798143 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.798121    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df730895-1ff7-47dc-b459-514ac4ec0c16-maas-api-tls\") pod \"maas-api-579fddb5f-jt9h2\" (UID: \"df730895-1ff7-47dc-b459-514ac4ec0c16\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:26.798309 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.798291    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-metrics-certs\") pod \"maas-api-7f5c8b48b7-bv67p\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:26.798417 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.798397    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-maas-api-tls\") pod \"maas-api-7f5c8b48b7-bv67p\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:26.802236 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.802199    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bgx6\" (UniqueName: \"kubernetes.io/projected/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-kube-api-access-8bgx6\") pod \"maas-api-7f5c8b48b7-bv67p\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:26.802355 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.802334    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrd2k\" (UniqueName: \"kubernetes.io/projected/df730895-1ff7-47dc-b459-514ac4ec0c16-kube-api-access-wrd2k\") pod \"maas-api-579fddb5f-jt9h2\" (UID: \"df730895-1ff7-47dc-b459-514ac4ec0c16\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:26.902127 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.902075    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:29:26.954502 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.954477    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:26.967610 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:26.967589    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:27.232528 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:27.232504    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"]
Sep 14 08:29:27.235232 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:29:27.235189    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6acf6ede_aaec_4ad0_8c36_9d3050741b1a.slice/crio-bc4cb99d4c865d4745813c19c88042d0b2fc00cab6df2aa5e2551303e534bb7b WatchSource:0}: Error finding container bc4cb99d4c865d4745813c19c88042d0b2fc00cab6df2aa5e2551303e534bb7b: Status 404 returned error can't find the container with id bc4cb99d4c865d4745813c19c88042d0b2fc00cab6df2aa5e2551303e534bb7b
Sep 14 08:29:27.236630 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:27.236613    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:29:27.251229 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:27.251194    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"]
Sep 14 08:29:27.252681 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:29:27.252655    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf730895_1ff7_47dc_b459_514ac4ec0c16.slice/crio-163c9eb8891deed0c2d752c4061a95fa1f2d49608e32e1d8cc03ce392672bf6a WatchSource:0}: Error finding container 163c9eb8891deed0c2d752c4061a95fa1f2d49608e32e1d8cc03ce392672bf6a: Status 404 returned error can't find the container with id 163c9eb8891deed0c2d752c4061a95fa1f2d49608e32e1d8cc03ce392672bf6a
Sep 14 08:29:27.418142 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:27.418109    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p" event={"ID":"6acf6ede-aaec-4ad0-8c36-9d3050741b1a","Type":"ContainerStarted","Data":"bc4cb99d4c865d4745813c19c88042d0b2fc00cab6df2aa5e2551303e534bb7b"}
Sep 14 08:29:27.419164 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:27.419141    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2" event={"ID":"df730895-1ff7-47dc-b459-514ac4ec0c16","Type":"ContainerStarted","Data":"163c9eb8891deed0c2d752c4061a95fa1f2d49608e32e1d8cc03ce392672bf6a"}
Sep 14 08:29:28.425887 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:28.425855    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p" event={"ID":"6acf6ede-aaec-4ad0-8c36-9d3050741b1a","Type":"ContainerStarted","Data":"172fe45486210e40ea7d5c5c9ed0234e197c0d4f916d68eb3d2c49c7d8d426e4"}
Sep 14 08:29:28.426343 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:28.425954    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:28.427401 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:28.427369    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2" event={"ID":"df730895-1ff7-47dc-b459-514ac4ec0c16","Type":"ContainerStarted","Data":"f88b46ea5fe0cb9999681d930877976f677762b01ca26a2a30c00c8fb1ec89c3"}
Sep 14 08:29:28.427505 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:28.427479    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:28.440908 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:28.440865    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p" podStartSLOduration=1.999887899 podStartE2EDuration="2.440851708s" totalImagesPullingTime="440.963809ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:29:28.439712922 +0000 UTC m=+2423.144251423" watchObservedRunningTime="2026-09-14 08:29:28.440851708 +0000 UTC m=+2423.145390209"
Sep 14 08:29:28.455542 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:28.455484    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2" podStartSLOduration=2.01947123 podStartE2EDuration="2.455467556s" totalImagesPullingTime="435.996326ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:29:28.454990621 +0000 UTC m=+2423.159529125" watchObservedRunningTime="2026-09-14 08:29:28.455467556 +0000 UTC m=+2423.160006059"
Sep 14 08:29:33.885364 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:33.885330    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:29:34.439827 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.439795    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-579fddb5f-jt9h2"
Sep 14 08:29:34.440106 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.440086    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:34.501441 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.501400    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"]
Sep 14 08:29:34.501658 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.501632    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p" podUID="6acf6ede-aaec-4ad0-8c36-9d3050741b1a" containerName="maas-api" containerID="cri-o://172fe45486210e40ea7d5c5c9ed0234e197c0d4f916d68eb3d2c49c7d8d426e4" gracePeriod=30
Sep 14 08:29:34.747608 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.747586    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:34.866616 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.866588    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-maas-api-tls\") pod \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") "
Sep 14 08:29:34.866745 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.866652    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-metrics-certs\") pod \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") "
Sep 14 08:29:34.866745 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.866676    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bgx6\" (UniqueName: \"kubernetes.io/projected/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-kube-api-access-8bgx6\") pod \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") "
Sep 14 08:29:34.866745 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.866720    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-openshift-service-ca\") pod \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\" (UID: \"6acf6ede-aaec-4ad0-8c36-9d3050741b1a\") "
Sep 14 08:29:34.867120 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.867095    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6acf6ede-aaec-4ad0-8c36-9d3050741b1a" (UID: "6acf6ede-aaec-4ad0-8c36-9d3050741b1a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:29:34.868621 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.868594    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6acf6ede-aaec-4ad0-8c36-9d3050741b1a" (UID: "6acf6ede-aaec-4ad0-8c36-9d3050741b1a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:29:34.868739 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.868725    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-kube-api-access-8bgx6" (OuterVolumeSpecName: "kube-api-access-8bgx6") pod "6acf6ede-aaec-4ad0-8c36-9d3050741b1a" (UID: "6acf6ede-aaec-4ad0-8c36-9d3050741b1a"). InnerVolumeSpecName "kube-api-access-8bgx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:29:34.868784 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.868726    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6acf6ede-aaec-4ad0-8c36-9d3050741b1a" (UID: "6acf6ede-aaec-4ad0-8c36-9d3050741b1a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:29:34.968157 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.968112    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-metrics-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:29:34.968157 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.968140    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bgx6\" (UniqueName: \"kubernetes.io/projected/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-kube-api-access-8bgx6\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:29:34.968157 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.968154    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-openshift-service-ca\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:29:34.968580 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:34.968168    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6acf6ede-aaec-4ad0-8c36-9d3050741b1a-maas-api-tls\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\""
Sep 14 08:29:35.471186 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:35.471144    2576 generic.go:356] "Generic (PLEG): container finished" podID="6acf6ede-aaec-4ad0-8c36-9d3050741b1a" containerID="172fe45486210e40ea7d5c5c9ed0234e197c0d4f916d68eb3d2c49c7d8d426e4" exitCode=0
Sep 14 08:29:35.471385 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:35.471224    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"
Sep 14 08:29:35.471385 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:35.471243    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p" event={"ID":"6acf6ede-aaec-4ad0-8c36-9d3050741b1a","Type":"ContainerDied","Data":"172fe45486210e40ea7d5c5c9ed0234e197c0d4f916d68eb3d2c49c7d8d426e4"}
Sep 14 08:29:35.471385 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:35.471342    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p" event={"ID":"6acf6ede-aaec-4ad0-8c36-9d3050741b1a","Type":"ContainerDied","Data":"bc4cb99d4c865d4745813c19c88042d0b2fc00cab6df2aa5e2551303e534bb7b"}
Sep 14 08:29:35.471385 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:35.471373    2576 scope.go:117] "RemoveContainer" containerID="172fe45486210e40ea7d5c5c9ed0234e197c0d4f916d68eb3d2c49c7d8d426e4"
Sep 14 08:29:35.481115 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:35.481092    2576 scope.go:117] "RemoveContainer" containerID="172fe45486210e40ea7d5c5c9ed0234e197c0d4f916d68eb3d2c49c7d8d426e4"
Sep 14 08:29:35.481430 ip-10-0-133-123 kubenswrapper[2576]: E0914 08:29:35.481412    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"172fe45486210e40ea7d5c5c9ed0234e197c0d4f916d68eb3d2c49c7d8d426e4\": container with ID starting with 172fe45486210e40ea7d5c5c9ed0234e197c0d4f916d68eb3d2c49c7d8d426e4 not found: ID does not exist" containerID="172fe45486210e40ea7d5c5c9ed0234e197c0d4f916d68eb3d2c49c7d8d426e4"
Sep 14 08:29:35.481516 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:35.481435    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172fe45486210e40ea7d5c5c9ed0234e197c0d4f916d68eb3d2c49c7d8d426e4"} err="failed to get container status \"172fe45486210e40ea7d5c5c9ed0234e197c0d4f916d68eb3d2c49c7d8d426e4\": rpc error: code = NotFound desc = could not find container \"172fe45486210e40ea7d5c5c9ed0234e197c0d4f916d68eb3d2c49c7d8d426e4\": container with ID starting with 172fe45486210e40ea7d5c5c9ed0234e197c0d4f916d68eb3d2c49c7d8d426e4 not found: ID does not exist"
Sep 14 08:29:35.492592 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:35.492564    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"]
Sep 14 08:29:35.494253 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:35.494232    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f5c8b48b7-bv67p"]
Sep 14 08:29:35.839262 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:35.839178    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6acf6ede-aaec-4ad0-8c36-9d3050741b1a" path="/var/lib/kubelet/pods/6acf6ede-aaec-4ad0-8c36-9d3050741b1a/volumes"
Sep 14 08:29:38.890573 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:38.890536    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:29:45.190326 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:45.190286    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:29:51.897466 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:29:51.897424    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gdlnh"]
Sep 14 08:30:00.604193 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:00.604152    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-tqvk8_f2b42013-7304-48c6-9016-5b64df1a32b4/manager/0.log"
Sep 14 08:30:01.225757 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:01.225727    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4cb54c55-qxz66_df73aa99-837f-4751-9b29-6143cf8ef9be/copy-manifests/0.log"
Sep 14 08:30:01.239752 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:01.239725    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4cb54c55-qxz66_df73aa99-837f-4751-9b29-6143cf8ef9be/manager/0.log"
Sep 14 08:30:01.830618 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:01.830587    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-579fddb5f-jt9h2_df730895-1ff7-47dc-b459-514ac4ec0c16/maas-api/0.log"
Sep 14 08:30:03.226538 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:03.226492    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-q2qzk_25e8a50a-1b49-4810-916e-d1e7e06ca1be/manager/0.log"
Sep 14 08:30:03.342764 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:03.342742    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-7rgrt_f8de7c18-9ea3-4fb9-a893-6d328dc4257a/manager/0.log"
Sep 14 08:30:03.690535 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:03.690504    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-gdlnh_f0f6364d-e5b4-4899-8ebd-2a2824c84534/limitador/0.log"
Sep 14 08:30:04.179629 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:04.179599    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp_930c3440-6e72-4f25-baf8-5fca805f538d/istio-proxy/0.log"
Sep 14 08:30:04.302830 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:04.302804    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hhp85_9f4d1bab-2cfb-4cba-995b-16f0463b3c37/discovery/0.log"
Sep 14 08:30:04.422250 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:04.422228    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fb8dbff8-bfs7h_e792ccde-27e1-4543-8252-0c5468bab8dc/kube-auth-proxy/0.log"
Sep 14 08:30:04.655357 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:04.655329    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-fct8p_11002022-86a1-4120-812b-41b43fd80b36/istio-proxy/1.log"
Sep 14 08:30:05.041724 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:05.041642    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-756cc58c7d-c8fsp_b7577aa4-cff4-4194-ad9d-6b7e6165413f/router/0.log"
Sep 14 08:30:05.524446 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:05.524418    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-q9498_624c18ed-3aef-4bb2-ad28-2c4ec5e3f308/main/0.log"
Sep 14 08:30:10.586393 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:10.586353    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:30:12.279413 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:12.279381    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:30:12.357234 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:12.357181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log" limit=1024
Sep 14 08:30:15.278621 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:15.278586    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fl64c_5271f263-1387-45f8-9d53-db6e1b9df066/global-pull-secret-syncer/0.log"
Sep 14 08:30:15.362612 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:15.362586    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6bfdg_55a0e6d8-3d6e-4bf6-b50f-5f5d5f94ac41/konnectivity-agent/0.log"
Sep 14 08:30:15.468973 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:15.468940    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-123.ec2.internal_8c553a2f2a397bf530bccc9a54adec3a/haproxy/0.log"
Sep 14 08:30:19.574906 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:19.574877    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-q2qzk_25e8a50a-1b49-4810-916e-d1e7e06ca1be/manager/0.log"
Sep 14 08:30:19.607741 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:19.607714    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-7rgrt_f8de7c18-9ea3-4fb9-a893-6d328dc4257a/manager/0.log"
Sep 14 08:30:19.706484 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:19.706462    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-gdlnh_f0f6364d-e5b4-4899-8ebd-2a2824c84534/limitador/0.log"
Sep 14 08:30:21.474838 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:21.474800    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-fwqng_e5088055-c0a7-475b-8808-129b29373f89/cluster-monitoring-operator/0.log"
Sep 14 08:30:21.501026 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:21.500998    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2mq8j_58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b/kube-state-metrics/0.log"
Sep 14 08:30:21.517024 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:21.516999    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2mq8j_58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b/kube-rbac-proxy-main/0.log"
Sep 14 08:30:21.537268 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:21.537239    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2mq8j_58cd67d9-571d-4ea3-bfa3-ccc5bf14e75b/kube-rbac-proxy-self/0.log"
Sep 14 08:30:21.587496 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:21.587468    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-9qc68_b1667a48-f833-4284-b7c3-c8fadef6b81d/monitoring-plugin/0.log"
Sep 14 08:30:21.701091 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:21.701059    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hkkt9_12caa7c9-5dfe-4da2-840e-227c707440e9/node-exporter/0.log"
Sep 14 08:30:21.718785 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:21.718757    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hkkt9_12caa7c9-5dfe-4da2-840e-227c707440e9/kube-rbac-proxy/0.log"
Sep 14 08:30:21.740969 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:21.740904    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hkkt9_12caa7c9-5dfe-4da2-840e-227c707440e9/init-textfile/0.log"
Sep 14 08:30:22.082910 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:22.082829    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-b4bg7_725b3545-1cdb-48bf-8a09-d20468e44525/prometheus-operator/0.log"
Sep 14 08:30:22.104451 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:22.104420    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-b4bg7_725b3545-1cdb-48bf-8a09-d20468e44525/kube-rbac-proxy/0.log"
Sep 14 08:30:23.872465 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:23.872439    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-pgvq5_8c3cbd65-3720-44ff-a4c5-d8b6e7a65dde/networking-console-plugin/0.log"
Sep 14 08:30:25.788429 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.788390    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"]
Sep 14 08:30:25.788874 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.788775    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6acf6ede-aaec-4ad0-8c36-9d3050741b1a" containerName="maas-api"
Sep 14 08:30:25.788874 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.788828    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6acf6ede-aaec-4ad0-8c36-9d3050741b1a" containerName="maas-api"
Sep 14 08:30:25.789046 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.788937    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6acf6ede-aaec-4ad0-8c36-9d3050741b1a" containerName="maas-api"
Sep 14 08:30:25.792250 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.792226    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:25.794606 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.794578    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bqggw\"/\"openshift-service-ca.crt\""
Sep 14 08:30:25.794730 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.794644    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bqggw\"/\"kube-root-ca.crt\""
Sep 14 08:30:25.794730 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.794668    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bqggw\"/\"default-dockercfg-frvbt\""
Sep 14 08:30:25.798548 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.798526    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"]
Sep 14 08:30:25.893500 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.893471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-sys\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:25.893500 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.893500    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-proc\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:25.893717 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.893544    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-lib-modules\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:25.893717 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.893618    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmdht\" (UniqueName: \"kubernetes.io/projected/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-kube-api-access-cmdht\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:25.893717 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.893689    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-podres\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:25.994308 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.994281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-sys\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:25.994308 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.994307    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-proc\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:25.994459 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.994333    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-lib-modules\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:25.994459 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.994359    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmdht\" (UniqueName: \"kubernetes.io/projected/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-kube-api-access-cmdht\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:25.994459 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.994393    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-podres\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:25.994459 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.994405    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-sys\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:25.994459 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.994419    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-proc\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:25.994618 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.994488    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-podres\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:25.994618 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:25.994490    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-lib-modules\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:26.002656 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:26.002634    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmdht\" (UniqueName: \"kubernetes.io/projected/618cba4f-cec4-4e30-8b2e-dd34a07bae3a-kube-api-access-cmdht\") pod \"perf-node-gather-daemonset-5qwp6\" (UID: \"618cba4f-cec4-4e30-8b2e-dd34a07bae3a\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:26.103858 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:26.103837    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:26.379020 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:26.378994    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"]
Sep 14 08:30:26.380836 ip-10-0-133-123 kubenswrapper[2576]: W0914 08:30:26.380809    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod618cba4f_cec4_4e30_8b2e_dd34a07bae3a.slice/crio-f70f31ec35dd6e07ae706942d8ff8c9064b4e748ac72ef2435af6706a7217f0c WatchSource:0}: Error finding container f70f31ec35dd6e07ae706942d8ff8c9064b4e748ac72ef2435af6706a7217f0c: Status 404 returned error can't find the container with id f70f31ec35dd6e07ae706942d8ff8c9064b4e748ac72ef2435af6706a7217f0c
Sep 14 08:30:26.711515 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:26.711411    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6" event={"ID":"618cba4f-cec4-4e30-8b2e-dd34a07bae3a","Type":"ContainerStarted","Data":"dfcd975cf3ed941b23104ccfc315531dc77fc7ce2f7958171e63bc4d2b0bedad"}
Sep 14 08:30:26.711515 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:26.711475    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6" event={"ID":"618cba4f-cec4-4e30-8b2e-dd34a07bae3a","Type":"ContainerStarted","Data":"f70f31ec35dd6e07ae706942d8ff8c9064b4e748ac72ef2435af6706a7217f0c"}
Sep 14 08:30:26.711709 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:26.711534    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:26.727386 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:26.727342    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6" podStartSLOduration=1.72732811 podStartE2EDuration="1.72732811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 08:30:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:30:26.725476753 +0000 UTC m=+2481.430015259" watchObservedRunningTime="2026-09-14 08:30:26.72732811 +0000 UTC m=+2481.431866608"
Sep 14 08:30:27.056830 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:27.056762    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pp6kf_719989fb-5c65-4237-9352-addb9eaf5da0/dns/0.log"
Sep 14 08:30:27.075393 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:27.075364    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pp6kf_719989fb-5c65-4237-9352-addb9eaf5da0/kube-rbac-proxy/0.log"
Sep 14 08:30:27.142279 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:27.142253    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vqbl2_bd19124b-82d8-4f39-a028-cfc81c6979c8/dns-node-resolver/0.log"
Sep 14 08:30:27.679356 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:27.679325    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7bc5c4ffc4-p2pqp_2355229a-ae08-4fb2-95bb-607e70de9907/registry/0.log"
Sep 14 08:30:27.718112 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:27.718091    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qgb6w_41a95c54-5096-456d-843c-62ee949876c2/node-ca/0.log"
Sep 14 08:30:28.619351 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:28.619319    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp_930c3440-6e72-4f25-baf8-5fca805f538d/istio-proxy/0.log"
Sep 14 08:30:29.027318 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:29.027239    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hhp85_9f4d1bab-2cfb-4cba-995b-16f0463b3c37/discovery/0.log"
Sep 14 08:30:29.045510 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:29.045486    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fb8dbff8-bfs7h_e792ccde-27e1-4543-8252-0c5468bab8dc/kube-auth-proxy/0.log"
Sep 14 08:30:29.096113 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:29.096091    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-fct8p_11002022-86a1-4120-812b-41b43fd80b36/istio-proxy/0.log"
Sep 14 08:30:29.098250 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:29.098220    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-fct8p_11002022-86a1-4120-812b-41b43fd80b36/istio-proxy/1.log"
Sep 14 08:30:29.185578 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:29.185556    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-756cc58c7d-c8fsp_b7577aa4-cff4-4194-ad9d-6b7e6165413f/router/0.log"
Sep 14 08:30:30.101667 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:30.101634    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n5qtb_6d25763c-a961-4ea4-8a12-70f9fc32945d/serve-healthcheck-canary/0.log"
Sep 14 08:30:30.709949 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:30.709923    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fb9mc_816becf2-a3af-484e-8a99-f99437f46b0c/kube-rbac-proxy/0.log"
Sep 14 08:30:30.727011 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:30.726986    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fb9mc_816becf2-a3af-484e-8a99-f99437f46b0c/exporter/0.log"
Sep 14 08:30:30.744787 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:30.744766    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fb9mc_816becf2-a3af-484e-8a99-f99437f46b0c/extractor/0.log"
Sep 14 08:30:32.728256 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:32.728222    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-5qwp6"
Sep 14 08:30:33.084257 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:33.084152    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-tqvk8_f2b42013-7304-48c6-9016-5b64df1a32b4/manager/0.log"
Sep 14 08:30:33.376547 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:33.376516    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4cb54c55-qxz66_df73aa99-837f-4751-9b29-6143cf8ef9be/manager/0.log"
Sep 14 08:30:33.392219 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:33.392182    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4cb54c55-qxz66_df73aa99-837f-4751-9b29-6143cf8ef9be/copy-manifests/0.log"
Sep 14 08:30:36.071734 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:36.071698    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp_930c3440-6e72-4f25-baf8-5fca805f538d/istio-proxy/0.log"
Sep 14 08:30:36.479098 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:36.479072    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hhp85_9f4d1bab-2cfb-4cba-995b-16f0463b3c37/discovery/0.log"
Sep 14 08:30:36.488708 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:36.488681    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fb8dbff8-bfs7h_e792ccde-27e1-4543-8252-0c5468bab8dc/kube-auth-proxy/0.log"
Sep 14 08:30:36.521335 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:36.521307    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-fct8p_11002022-86a1-4120-812b-41b43fd80b36/istio-proxy/0.log"
Sep 14 08:30:36.524151 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:36.524125    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-fct8p_11002022-86a1-4120-812b-41b43fd80b36/istio-proxy/1.log"
Sep 14 08:30:36.589679 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:36.589646    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-756cc58c7d-c8fsp_b7577aa4-cff4-4194-ad9d-6b7e6165413f/router/0.log"
Sep 14 08:30:38.178192 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:38.178134    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799ckjfdp_930c3440-6e72-4f25-baf8-5fca805f538d/istio-proxy/0.log"
Sep 14 08:30:38.586462 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:38.586390    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hhp85_9f4d1bab-2cfb-4cba-995b-16f0463b3c37/discovery/0.log"
Sep 14 08:30:38.606461 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:38.606434    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fb8dbff8-bfs7h_e792ccde-27e1-4543-8252-0c5468bab8dc/kube-auth-proxy/0.log"
Sep 14 08:30:38.642077 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:38.642053    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-fct8p_11002022-86a1-4120-812b-41b43fd80b36/istio-proxy/0.log"
Sep 14 08:30:38.644282 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:38.644263    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-fct8p_11002022-86a1-4120-812b-41b43fd80b36/istio-proxy/1.log"
Sep 14 08:30:38.720230 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:38.720183    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-756cc58c7d-c8fsp_b7577aa4-cff4-4194-ad9d-6b7e6165413f/router/0.log"
Sep 14 08:30:40.185629 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:40.185585    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-jtrdh_bea0a7a6-e3aa-4c65-a748-a93ba8ea5989/migrator/0.log"
Sep 14 08:30:40.204938 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:40.204892    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-jtrdh_bea0a7a6-e3aa-4c65-a748-a93ba8ea5989/graceful-termination/0.log"
Sep 14 08:30:41.715539 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:41.715464    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7zkjj_c850c672-bf8a-4935-830c-db4cd90d3c6b/kube-multus-additional-cni-plugins/0.log"
Sep 14 08:30:41.734624 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:41.734601    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7zkjj_c850c672-bf8a-4935-830c-db4cd90d3c6b/egress-router-binary-copy/0.log"
Sep 14 08:30:41.752004 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:41.751979    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7zkjj_c850c672-bf8a-4935-830c-db4cd90d3c6b/cni-plugins/0.log"
Sep 14 08:30:41.769059 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:41.769039    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7zkjj_c850c672-bf8a-4935-830c-db4cd90d3c6b/bond-cni-plugin/0.log"
Sep 14 08:30:41.793554 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:41.793530    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7zkjj_c850c672-bf8a-4935-830c-db4cd90d3c6b/routeoverride-cni/0.log"
Sep 14 08:30:41.815107 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:41.815086    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7zkjj_c850c672-bf8a-4935-830c-db4cd90d3c6b/whereabouts-cni-bincopy/0.log"
Sep 14 08:30:41.832177 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:41.832155    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7zkjj_c850c672-bf8a-4935-830c-db4cd90d3c6b/whereabouts-cni/0.log"
Sep 14 08:30:42.068352 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:42.068279    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pwcsr_505f1fe1-8e14-4b98-b9b8-9a90faf97b5d/kube-multus/0.log"
Sep 14 08:30:42.216713 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:42.216689    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pwczh_89ce251d-66b1-4990-8d8a-101dafeb009b/network-metrics-daemon/0.log"
Sep 14 08:30:42.234045 ip-10-0-133-123 kubenswrapper[2576]: I0914 08:30:42.234019    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pwczh_89ce251d-66b1-4990-8d8a-101dafeb009b/kube-rbac-proxy/0.log"