Sep 06 04:55:32.517315 ip-10-0-134-29 systemd[1]: Starting Kubernetes Kubelet...
Sep 06 04:55:32.945706 ip-10-0-134-29 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 04:55:32.945706 ip-10-0-134-29 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 06 04:55:32.945706 ip-10-0-134-29 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 04:55:32.945706 ip-10-0-134-29 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 06 04:55:32.945706 ip-10-0-134-29 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 04:55:32.946607 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.946522    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 06 04:55:32.950686 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950672    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 04:55:32.950686 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950687    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950691    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950694    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950699    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950701    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950704    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950707    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950710    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950712    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950715    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950718    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950720    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950723    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950726    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950729    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950731    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950734    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950738    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950741    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 04:55:32.950756 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950750    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950753    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950756    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950759    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950762    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950764    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950767    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950770    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950773    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950775    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950778    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950781    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950783    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950786    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950788    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950792    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950795    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950799    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950803    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 04:55:32.951230 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950806    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950808    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950811    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950813    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950816    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950818    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950821    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950823    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950826    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950828    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950831    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950834    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950836    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950839    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950841    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950846    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950849    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950851    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950854    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 04:55:32.951688 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950857    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950859    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950862    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950864    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950867    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950869    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950872    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950875    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950877    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950879    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950882    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950884    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950887    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950889    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950892    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950894    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950897    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950899    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950902    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950905    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 04:55:32.952181 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950909    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950911    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950914    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950917    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950919    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950922    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950932    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950935    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950938    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950941    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950943    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950946    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950948    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950951    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950954    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950957    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950977    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950981    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950985    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950987    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 04:55:32.952733 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.950990    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 04:55:32.955609 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955596    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 04:55:32.955609 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955608    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955612    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955616    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955619    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955621    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955624    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955627    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955630    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955632    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955635    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955638    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955641    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955644    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955646    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955649    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955652    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955655    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955657    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955660    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 04:55:32.955671 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955663    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955665    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955668    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955671    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955673    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955676    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955679    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955682    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955684    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955687    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955690    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955693    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955695    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955698    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955700    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955703    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955707    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955711    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955714    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 04:55:32.956160 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955716    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955719    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955722    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955725    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955728    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955731    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955734    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955737    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955739    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955742    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955744    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955747    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955749    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955752    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955754    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955757    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955760    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955763    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955766    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955768    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 04:55:32.956622 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955771    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955774    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955776    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955779    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955782    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955784    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955787    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955790    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955792    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955795    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955798    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955800    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955803    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955806    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955808    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955811    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955814    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955817    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955821    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955823    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 04:55:32.957104 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955825    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955828    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955831    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955833    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955836    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955838    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955841    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955845    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955848    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955852    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955855    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955859    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955861    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955864    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955866    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955869    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955871    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955875    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955878    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 04:55:32.957576 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.955882    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956284    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956289    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956292    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956296    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956299    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956303    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956306    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956308    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956311    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956313    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956317    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956320    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956322    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956325    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956327    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956330    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956332    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956335    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956337    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 04:55:32.958027 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956343    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956345    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956348    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956351    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956353    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956356    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956358    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956361    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956365    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956368    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956370    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956373    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956375    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956378    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956380    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956383    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956385    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956388    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956390    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956393    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 04:55:32.958519 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956395    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956398    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956400    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956403    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956406    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956409    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956412    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956414    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956416    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956419    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956421    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956424    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956429    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956432    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956434    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956436    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956439    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956441    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956444    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956447    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 04:55:32.959040 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956449    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956452    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956454    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956457    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956459    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956462    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956464    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956466    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956469    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956471    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956473    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956476    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956479    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956481    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956484    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956486    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956489    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956492    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956494    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956497    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 04:55:32.959523 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956499    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956502    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956506    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956509    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956513    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956516    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956519    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956522    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956525    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956528    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956530    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956533    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956536    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956538    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956541    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956544    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956546    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956549    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956551    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 04:55:32.960098 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.956554    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956624    2578 flags.go:64] FLAG: --address="0.0.0.0"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956671    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956678    2578 flags.go:64] FLAG: --anonymous-auth="true"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956682    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956686    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956690    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956694    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956699    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956702    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956705    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956709    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956712    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956715    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956718    2578 flags.go:64] FLAG: --cgroup-root=""
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956721    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956725    2578 flags.go:64] FLAG: --client-ca-file=""
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956728    2578 flags.go:64] FLAG: --cloud-provider="external"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956731    2578 flags.go:64] FLAG: --cluster-dns="[]"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956735    2578 flags.go:64] FLAG: --cluster-domain=""
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956737    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956740    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956743    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956747    2578 flags.go:64] FLAG: --container-log-max-files="5"
Sep 06 04:55:32.960554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956750    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956753    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956756    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956759    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956763    2578 flags.go:64] FLAG: --contention-profiling="false"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956765    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956768    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956772    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956775    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956778    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956781    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956784    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956787    2578 flags.go:64] FLAG: --enable-load-reader="false"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956790    2578 flags.go:64] FLAG: --enable-server="true"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956793    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956797    2578 flags.go:64] FLAG: --event-burst="100"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956802    2578 flags.go:64] FLAG: --event-qps="50"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956806    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956809    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956812    2578 flags.go:64] FLAG: --eviction-hard=""
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956816    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956819    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956822    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956825    2578 flags.go:64] FLAG: --eviction-soft=""
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956828    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 06 04:55:32.961106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956831    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956834    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956837    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956841    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956843    2578 flags.go:64] FLAG: --fail-swap-on="true"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956846    2578 flags.go:64] FLAG: --feature-gates=""
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956850    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956853    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956856    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956859    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956862    2578 flags.go:64] FLAG: --healthz-port="10248"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956865    2578 flags.go:64] FLAG: --help="false"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956868    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-134-29.ec2.internal"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956871    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956874    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956877    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956880    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956883    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956886    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956889    2578 flags.go:64] FLAG: --image-service-endpoint=""
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956892    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956895    2578 flags.go:64] FLAG: --kube-api-burst="100"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956898    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956904    2578 flags.go:64] FLAG: --kube-api-qps="50"
Sep 06 04:55:32.961740 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956906    2578 flags.go:64] FLAG: --kube-reserved=""
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956909    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956912    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956915    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956919    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956922    2578 flags.go:64] FLAG: --lock-file=""
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956925    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956927    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956930    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956936    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956939    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956941    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956944    2578 flags.go:64] FLAG: --logging-format="text"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956947    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956950    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956953    2578 flags.go:64] FLAG: --manifest-url=""
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956956    2578 flags.go:64] FLAG: --manifest-url-header=""
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956960    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956963    2578 flags.go:64] FLAG: --max-open-files="1000000"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956967    2578 flags.go:64] FLAG: --max-pods="110"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956970    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956973    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956976    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956979    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956982    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 06 04:55:32.962326 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956985    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956988    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956995    2578 flags.go:64] FLAG: --node-status-max-images="50"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.956998    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957001    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957004    2578 flags.go:64] FLAG: --pod-cidr=""
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957009    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957025    2578 flags.go:64] FLAG: --pod-manifest-path=""
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957028    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957031    2578 flags.go:64] FLAG: --pods-per-core="0"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957035    2578 flags.go:64] FLAG: --port="10250"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957038    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957042    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e5809472684cbd47"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957045    2578 flags.go:64] FLAG: --qos-reserved=""
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957048    2578 flags.go:64] FLAG: --read-only-port="10255"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957051    2578 flags.go:64] FLAG: --register-node="true"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957054    2578 flags.go:64] FLAG: --register-with-taints=""
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957057    2578 flags.go:64] FLAG: --registry-burst="10"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957060    2578 flags.go:64] FLAG: --registry-qps="5"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957062    2578 flags.go:64] FLAG: --reserved-cpus=""
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957065    2578 flags.go:64] FLAG: --reserved-memory=""
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957069    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957072    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957075    2578 flags.go:64] FLAG: --rotate-certificates="false"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957078    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 06 04:55:32.962912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957081    2578 flags.go:64] FLAG: --runonce="false"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957084    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957087    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957090    2578 flags.go:64] FLAG: --seccomp-default="false"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957093    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957095    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957098    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957101    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957104    2578 flags.go:64] FLAG: --storage-driver-password="root"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957107    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957111    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957114    2578 flags.go:64] FLAG: --storage-driver-user="root"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957117    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957120    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957124    2578 flags.go:64] FLAG: --system-cgroups=""
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957126    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957132    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957134    2578 flags.go:64] FLAG: --tls-cert-file=""
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957137    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957141    2578 flags.go:64] FLAG: --tls-min-version=""
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957145    2578 flags.go:64] FLAG: --tls-private-key-file=""
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957148    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957151    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957154    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957156    2578 flags.go:64] FLAG: --v="2"
Sep 06 04:55:32.963567 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957160    2578 flags.go:64] FLAG: --version="false"
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957164    2578 flags.go:64] FLAG: --vmodule=""
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957167    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.957170    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957281    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957284    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957289    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957291    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957294    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957296    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957299    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957302    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957304    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957307    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957309    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957312    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957315    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957317    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957320    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957322    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957325    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 04:55:32.964202 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957329    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957331    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957334    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957336    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957339    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957341    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957343    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957346    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957349    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957352    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957354    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957357    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957359    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957362    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957364    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957367    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957369    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957373    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957375    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 04:55:32.964770 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957377    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957380    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957383    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957385    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957387    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957390    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957392    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957395    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957399    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957403    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957405    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957408    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957410    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957414    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957417    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957419    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957422    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957424    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957427    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957429    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 04:55:32.965245 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957432    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957435    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957437    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957440    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957442    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957445    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957448    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957452    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957455    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957457    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957461    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957463    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957465    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957468    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957470    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957473    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957476    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957478    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957481    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957483    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 04:55:32.965712 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957486    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957488    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957491    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957493    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957496    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957500    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957502    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957505    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957507    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957510    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957512    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957515    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957517    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957520    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957523    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957526    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957528    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957531    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957533    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 04:55:32.966196 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957536    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 04:55:32.966634 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957538    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 04:55:32.966634 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957540    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 04:55:32.966634 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.957545    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 04:55:32.966634 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.958169    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 04:55:32.966634 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.965716    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 06 04:55:32.966634 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.965730    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 06 04:55:32.966634 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.965747    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 06 04:55:32.966634 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.965753    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 06 04:55:32.966634 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965810    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 04:55:32.966634 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965815    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 04:55:32.966634 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965819    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 04:55:32.966634 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965821    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 04:55:32.966634 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965825    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 04:55:32.966634 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965827    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965830    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965832    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965835    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965838    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965840    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965843    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965845    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965848    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965851    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965853    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965856    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965858    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965860    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965863    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965865    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965868    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965871    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965873    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965876    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 04:55:32.966998 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965880    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965883    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965886    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965888    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965892    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965895    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965897    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965900    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965902    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965905    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965907    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965910    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965912    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965914    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965917    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965919    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965922    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965925    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965927    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965929    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 04:55:32.967502 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965932    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965935    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965938    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965940    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965942    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965945    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965948    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965950    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965952    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965955    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965957    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965960    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965963    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965965    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965968    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965970    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965973    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965977    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965979    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965982    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 04:55:32.967986 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965984    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965987    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965989    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965992    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965994    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965997    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.965999    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966002    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966004    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966007    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966011    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966029    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966033    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966036    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966039    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966042    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966045    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966047    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966049    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 04:55:32.968550 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966052    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966054    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966057    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966059    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966062    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966064    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966067    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966071    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966074    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966078    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966081    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966085    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966088    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966090    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966093    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 04:55:32.968991 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.966098    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966208    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966213    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966216    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966219    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966221    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966224    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966227    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966229    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966232    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966234    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966237    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966239    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966242    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966245    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966247    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966250    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966252    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966255    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 04:55:32.969403 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966257    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966260    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966262    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966264    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966267    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966269    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966272    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966274    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966277    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966280    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966283    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966287    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966290    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966293    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966295    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966298    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966301    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966304    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966307    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 04:55:32.969844 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966310    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966312    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966315    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966317    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966320    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966322    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966325    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966327    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966330    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966332    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966334    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966337    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966339    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966342    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966344    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966347    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966349    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966351    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966354    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966356    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 04:55:32.970325 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966359    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966361    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966364    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966366    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966369    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966372    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966374    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966376    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966379    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966381    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966384    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966386    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966388    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966391    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966393    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966396    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966398    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966400    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966403    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966405    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 04:55:32.970795 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966408    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966410    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966412    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966415    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966417    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966420    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966422    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966424    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966427    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966429    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966432    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966434    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966436    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966440    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966443    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966446    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966449    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966452    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966454    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966457    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 04:55:32.971317 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966459    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 04:55:32.971798 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:32.966462    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 04:55:32.971798 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.966466    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 04:55:32.971798 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.966566    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 06 04:55:32.971798 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.968963    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 06 04:55:32.971798 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.970072    2578 server.go:1019] "Starting client certificate rotation"
Sep 06 04:55:32.971798 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.970168    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 06 04:55:32.971798 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.970227    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 06 04:55:32.972845 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.972832    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 06 04:55:32.974300 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.974282    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 06 04:55:32.981745 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.981728    2578 log.go:25] "Validated CRI v1 runtime API"
Sep 06 04:55:32.985851 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.985828    2578 log.go:25] "Validated CRI v1 image API"
Sep 06 04:55:32.986653 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.986638    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 06 04:55:32.989904 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.989884    2578 fs.go:135] Filesystem UUIDs: map[2857fcd0-67a2-4e4e-9e58-62946e9d11e8:/dev/nvme0n1p3 49e21662-532a-4388-b8e4-96ace0d1bc80:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 06 04:55:32.989960 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.989904    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 06 04:55:32.996402 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.996250    2578 manager.go:216] Machine: {Timestamp:2026-09-06 04:55:32.99481386 +0000 UTC m=+0.364269096 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3195918 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23997aef65173fb2d6beadef1a371a SystemUUID:ec23997a-ef65-173f-b2d6-beadef1a371a BootID:a1f49f94-9de4-4eb6-8ccb-f3fb456e7acb Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:71:38:fc:32:8f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:71:38:fc:32:8f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:d6:45:92:cf:11 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 06 04:55:32.996402 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.996397    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 06 04:55:32.996498 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:32.996488    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 06 04:55:33.000152 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.000122    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 06 04:55:33.000302 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.000152    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-29.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 06 04:55:33.000953 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.000938    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 06 04:55:33.001722 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.001711    2578 topology_manager.go:138] "Creating topology manager with none policy"
Sep 06 04:55:33.001754 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.001726    2578 container_manager_linux.go:309] "Creating device plugin manager"
Sep 06 04:55:33.001754 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.001734    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 06 04:55:33.001754 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.001748    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 06 04:55:33.003775 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.003761    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 06 04:55:33.008418 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.008397    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 06 04:55:33.008534 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.008525    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 06 04:55:33.008569 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.008554    2578 kubelet.go:486] "Attempting to sync node with API server"
Sep 06 04:55:33.008569 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.008563    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 06 04:55:33.008622 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.008577    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 06 04:55:33.008622 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.008586    2578 kubelet.go:398] "Adding apiserver pod source"
Sep 06 04:55:33.008622 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.008595    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 06 04:55:33.008959 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.008946    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 06 04:55:33.008991 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.008969    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 06 04:55:33.011297 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.011281    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 06 04:55:33.013592 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.013578    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 06 04:55:33.013639 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.013609    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 06 04:55:33.013746 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.013737    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 06 04:55:33.013781 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.013752    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 06 04:55:33.013781 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.013758    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 06 04:55:33.013781 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.013765    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 06 04:55:33.013781 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.013773    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 06 04:55:33.013781 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.013778    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 06 04:55:33.014421 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.014412    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 06 04:55:33.014455 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.014424    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 06 04:55:33.014455 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.014434    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 06 04:55:33.014455 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.014442    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 06 04:55:33.014533 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.014462    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 06 04:55:33.014533 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.014473    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 06 04:55:33.016301 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.016290    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 06 04:55:33.016301 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.016301    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 06 04:55:33.017069 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.017044    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 06 04:55:33.017113 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.017070    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-29.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 06 04:55:33.019999 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.019987    2578 server.go:1268] "Started kubelet"
Sep 06 04:55:33.020108 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.020072    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 06 04:55:33.020827 ip-10-0-134-29 systemd[1]: Started Kubernetes Kubelet.
Sep 06 04:55:33.024771 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.024396    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 06 04:55:33.024771 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.024578    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 06 04:55:33.025755 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.025728    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tlkpj"
Sep 06 04:55:33.026050 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.026011    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 06 04:55:33.026987 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.026937    2578 server.go:310] "Adding debug handlers to kubelet server"
Sep 06 04:55:33.027089 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.027063    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 06 04:55:33.027144 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.027116    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-29.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 06 04:55:33.028498 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.028482    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tlkpj"
Sep 06 04:55:33.032550 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.032535    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 06 04:55:33.032550 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.032548    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 06 04:55:33.032664 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.032643    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-29.ec2.internal\" not found"
Sep 06 04:55:33.033295 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.033275    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 06 04:55:33.033382 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.033277    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 06 04:55:33.033382 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.033317    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 06 04:55:33.033490 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.033411    2578 reconstruct.go:97] "Volume reconstruction finished"
Sep 06 04:55:33.033490 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.033420    2578 reconciler.go:29] "Reconciler: start to sync state"
Sep 06 04:55:33.033490 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.033439    2578 factory.go:55] Registering systemd factory
Sep 06 04:55:33.033490 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.033467    2578 factory.go:223] Registration of the systemd container factory successfully
Sep 06 04:55:33.033827 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.033807    2578 factory.go:153] Registering CRI-O factory
Sep 06 04:55:33.033827 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.033824    2578 factory.go:223] Registration of the crio container factory successfully
Sep 06 04:55:33.033920 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.033869    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 06 04:55:33.033920 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.033885    2578 factory.go:103] Registering Raw factory
Sep 06 04:55:33.033920 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.033898    2578 manager.go:1196] Started watching for new ooms in manager
Sep 06 04:55:33.034269 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.034256    2578 manager.go:349] Starting recovery of all containers
Sep 06 04:55:33.035652 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.035448    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-29.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 06 04:55:33.035652 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.035509    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 06 04:55:33.036738 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.035586    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-29.ec2.internal.18d2a25f7cdd0239  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-29.ec2.internal,UID:ip-10-0-134-29.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-29.ec2.internal,},FirstTimestamp:2026-09-06 04:55:33.019959865 +0000 UTC m=+0.389415102,LastTimestamp:2026-09-06 04:55:33.019959865 +0000 UTC m=+0.389415102,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-29.ec2.internal,}"
Sep 06 04:55:33.043204 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.043066    2578 manager.go:354] Recovery completed
Sep 06 04:55:33.047201 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.047187    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 04:55:33.049232 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.049216    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 04:55:33.049297 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.049248    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 04:55:33.049297 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.049260    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasSufficientPID"
Sep 06 04:55:33.049739 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.049723    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 06 04:55:33.049739 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.049735    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 06 04:55:33.049873 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.049747    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 04:55:33.049873 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.049749    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 06 04:55:33.050483 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.050466    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 04:55:33.050563 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.050495    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 04:55:33.050563 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.050508    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasSufficientPID"
Sep 06 04:55:33.051778 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.051759    2578 policy_none.go:49] "None policy: Start"
Sep 06 04:55:33.051778 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.051777    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 06 04:55:33.051905 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.051786    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 06 04:55:33.053819 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.053806    2578 policy_none.go:47] "Start"
Sep 06 04:55:33.086651 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.086636    2578 manager.go:337] "Starting Device Plugin manager"
Sep 06 04:55:33.107583 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.086681    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 06 04:55:33.107583 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.086695    2578 server.go:87] "Starting device plugin registration server"
Sep 06 04:55:33.107583 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.087001    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 06 04:55:33.107583 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.087038    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 06 04:55:33.107583 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.087121    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 06 04:55:33.107583 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.087195    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 06 04:55:33.107583 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.087207    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 06 04:55:33.107583 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.090650    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 06 04:55:33.107583 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.090702    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-29.ec2.internal\" not found"
Sep 06 04:55:33.145501 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.145475    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 06 04:55:33.146648 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.146632    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 06 04:55:33.146738 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.146686    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 06 04:55:33.146738 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.146715    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 06 04:55:33.146825 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.146759    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 06 04:55:33.149051 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.149031    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 04:55:33.187972 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.187943    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 04:55:33.188847 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.188834    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 04:55:33.188908 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.188861    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 04:55:33.188908 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.188872    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasSufficientPID"
Sep 06 04:55:33.188908 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.188894    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.196993 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.196929    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.196993 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.196950    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-29.ec2.internal\": node \"ip-10-0-134-29.ec2.internal\" not found"
Sep 06 04:55:33.209442 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.209421    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-29.ec2.internal\" not found"
Sep 06 04:55:33.247360 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.247332    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-29.ec2.internal"]
Sep 06 04:55:33.247501 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.247397    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 04:55:33.249011 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.248993    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 04:55:33.249111 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.249043    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 04:55:33.249111 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.249058    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasSufficientPID"
Sep 06 04:55:33.251857 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.251845    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 04:55:33.251972 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.251959    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.252008 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.251986    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 04:55:33.252558 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.252542    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 04:55:33.252558 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.252549    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 04:55:33.252654 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.252569    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 04:55:33.252654 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.252572    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 04:55:33.252654 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.252578    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasSufficientPID"
Sep 06 04:55:33.252654 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.252586    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasSufficientPID"
Sep 06 04:55:33.254778 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.254766    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.254835 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.254789    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 04:55:33.255461 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.255445    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 04:55:33.255526 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.255474    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 04:55:33.255526 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.255488    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeHasSufficientPID"
Sep 06 04:55:33.282565 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.282542    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-29.ec2.internal\" not found" node="ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.287111 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.287097    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-29.ec2.internal\" not found" node="ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.310434 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.310410    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-29.ec2.internal\" not found"
Sep 06 04:55:33.335339 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.335310    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9ed8709ffad4ba828dceaf06e7817a74-config\") pod \"kube-apiserver-proxy-ip-10-0-134-29.ec2.internal\" (UID: \"9ed8709ffad4ba828dceaf06e7817a74\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.335466 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.335341    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a5564bf9fd0f70518b201a8b4b15358-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal\" (UID: \"6a5564bf9fd0f70518b201a8b4b15358\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.335466 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.335366    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a5564bf9fd0f70518b201a8b4b15358-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal\" (UID: \"6a5564bf9fd0f70518b201a8b4b15358\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.411048 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.411009    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-29.ec2.internal\" not found"
Sep 06 04:55:33.436429 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.436394    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a5564bf9fd0f70518b201a8b4b15358-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal\" (UID: \"6a5564bf9fd0f70518b201a8b4b15358\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.436531 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.436439    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a5564bf9fd0f70518b201a8b4b15358-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal\" (UID: \"6a5564bf9fd0f70518b201a8b4b15358\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.436531 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.436466    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9ed8709ffad4ba828dceaf06e7817a74-config\") pod \"kube-apiserver-proxy-ip-10-0-134-29.ec2.internal\" (UID: \"9ed8709ffad4ba828dceaf06e7817a74\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.436531 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.436499    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a5564bf9fd0f70518b201a8b4b15358-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal\" (UID: \"6a5564bf9fd0f70518b201a8b4b15358\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.436531 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.436520    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9ed8709ffad4ba828dceaf06e7817a74-config\") pod \"kube-apiserver-proxy-ip-10-0-134-29.ec2.internal\" (UID: \"9ed8709ffad4ba828dceaf06e7817a74\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.436664 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.436536    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a5564bf9fd0f70518b201a8b4b15358-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal\" (UID: \"6a5564bf9fd0f70518b201a8b4b15358\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.511853 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.511796    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-29.ec2.internal\" not found"
Sep 06 04:55:33.585329 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.585298    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.589895 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.589859    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:33.613602 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.613570    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-29.ec2.internal\" not found"
Sep 06 04:55:33.714099 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.714063    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-29.ec2.internal\" not found"
Sep 06 04:55:33.814460 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.814401    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-29.ec2.internal\" not found"
Sep 06 04:55:33.914560 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:33.914529    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-29.ec2.internal\" not found"
Sep 06 04:55:33.956235 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.956206    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 04:55:33.969991 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.969969    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 06 04:55:33.970121 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.970102    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 06 04:55:33.970185 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:33.970137    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 06 04:55:34.009155 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.009126    2578 apiserver.go:52] "Watching apiserver"
Sep 06 04:55:34.026631 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.026612    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 06 04:55:34.028612 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.028592    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-jgkf2","openshift-multus/multus-j6jvp","openshift-multus/network-metrics-daemon-dxn9s","openshift-network-diagnostics/network-check-target-k5krn","kube-system/global-pull-secret-syncer-n2jlj","openshift-cluster-node-tuning-operator/tuned-w895c","openshift-dns/node-resolver-79mhk","openshift-network-operator/iptables-alerter-4qwvq","openshift-ovn-kubernetes/ovnkube-node-zfz7w","kube-system/konnectivity-agent-4bhll","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj","openshift-image-registry/node-ca-97hkw"]
Sep 06 04:55:34.031262 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.031246    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.033224 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.033200    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 06 04:55:34.033318 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.033208    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:34.033510 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.033490    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.033583 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.033546    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 06 04:55:34.033635 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.033550    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 06 04:55:34.034622 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.034570    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-05 04:50:33 +0000 UTC" deadline="2028-03-21 14:21:34.418530245 +0000 UTC"
Sep 06 04:55:34.034622 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.034620    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13497h26m0.383914489s"
Sep 06 04:55:34.034995 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.034976    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 06 04:55:34.035080 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.035048    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 06 04:55:34.035204 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.035180    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 06 04:55:34.035807 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.035788    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-425w4\""
Sep 06 04:55:34.036960 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.036939    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:34.037105 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.037074    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:55:34.037646 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.037619    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 06 04:55:34.037813 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.037795    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-chmnx\""
Sep 06 04:55:34.039195 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039175    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-cni-binary-copy\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.039291 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039205    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-var-lib-cni-bin\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.039291 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039229    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-multus-conf-dir\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.039291 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039251    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-etc-kubernetes\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.039291 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039276    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-cnibin\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.039473 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039300    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.039473 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039336    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.039473 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039372    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-multus-cni-dir\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.039473 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039390    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-multus-socket-dir-parent\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.039473 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039406    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-run-netns\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.039473 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039430    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/837ae00e-1c4e-4e9c-8170-d7c018203862-multus-daemon-config\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.039473 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039457    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zsnn\" (UniqueName: \"kubernetes.io/projected/837ae00e-1c4e-4e9c-8170-d7c018203862-kube-api-access-9zsnn\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.039473 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039461    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:34.039755 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039510    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.039755 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.039520    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:55:34.039755 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039560    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghrkf\" (UniqueName: \"kubernetes.io/projected/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-kube-api-access-ghrkf\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.039755 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039587    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-system-cni-dir\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.039755 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039612    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/837ae00e-1c4e-4e9c-8170-d7c018203862-cni-binary-copy\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.039755 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039636    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-var-lib-cni-multus\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.039755 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039657    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-var-lib-kubelet\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.039755 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039676    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-hostroot\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.039755 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-system-cni-dir\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.039755 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039735    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-cnibin\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.039755 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039756    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-run-k8s-cni-cncf-io\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.040069 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.039771    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-run-multus-certs\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.041813 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.041796    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:34.041896 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.041872    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:55:34.044005 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.043988    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.045954 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.045929    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 06 04:55:34.046027 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.045968    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4tnz5\""
Sep 06 04:55:34.046027 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.045973    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 06 04:55:34.046766 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.046750    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 06 04:55:34.046837 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.046830    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-29.ec2.internal"
Sep 06 04:55:34.047100 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.047078    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-79mhk"
Sep 06 04:55:34.048684 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.048666    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 06 04:55:34.049227 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.049209    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 06 04:55:34.049314 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.049273    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xtxlb\""
Sep 06 04:55:34.049395 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.049380    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 06 04:55:34.049647 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.049623    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4qwvq"
Sep 06 04:55:34.051589 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.051573    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 06 04:55:34.051756 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.051740    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 06 04:55:34.051881 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.051853    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 06 04:55:34.051942 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.051856    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jmthr\""
Sep 06 04:55:34.052252 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.052237    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.054807 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.054661    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 06 04:55:34.054807 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.054669    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7sjxc\""
Sep 06 04:55:34.054807 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.054683    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 06 04:55:34.054807 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.054684    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 06 04:55:34.054807 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.054688    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 06 04:55:34.055164 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.055015    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4bhll"
Sep 06 04:55:34.055164 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.055114    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 06 04:55:34.055164 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.055124    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 06 04:55:34.055994 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.055979    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 06 04:55:34.057189 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.057175    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 06 04:55:34.057353 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.057339    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 06 04:55:34.057404 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.057340    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5xbtf\""
Sep 06 04:55:34.057474 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.057458    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.059524 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.059502    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 06 04:55:34.059649 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.059629    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7b59q\""
Sep 06 04:55:34.059717 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.059649    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 06 04:55:34.059757 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.059632    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 06 04:55:34.060063 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.060033    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal"]
Sep 06 04:55:34.060119 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.060078    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-29.ec2.internal"]
Sep 06 04:55:34.060166 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.060155    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-97hkw"
Sep 06 04:55:34.064267 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.064252    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 06 04:55:34.064488 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.064448    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 06 04:55:34.064717 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.064700    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lxvk4\""
Sep 06 04:55:34.064802 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.064742    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 06 04:55:34.068736 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.068720    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6527c"
Sep 06 04:55:34.084660 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.084644    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6527c"
Sep 06 04:55:34.134990 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.134966    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 06 04:55:34.140550 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140525    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:34.140698 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140559    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.140698 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140580    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zsnn\" (UniqueName: \"kubernetes.io/projected/837ae00e-1c4e-4e9c-8170-d7c018203862-kube-api-access-9zsnn\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.140698 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140601    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtgxw\" (UniqueName: \"kubernetes.io/projected/ed8f6484-9497-4655-853a-1b79956c4145-kube-api-access-mtgxw\") pod \"node-ca-97hkw\" (UID: \"ed8f6484-9497-4655-853a-1b79956c4145\") " pod="openshift-image-registry/node-ca-97hkw"
Sep 06 04:55:34.140698 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140626    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.140698 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140657    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-ovnkube-config\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.140698 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140700    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-system-cni-dir\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.140989 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140727    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/837ae00e-1c4e-4e9c-8170-d7c018203862-cni-binary-copy\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.140989 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140752    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-var-lib-cni-multus\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.140989 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140774    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-hostroot\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.140989 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140798    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-run-netns\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.140989 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140822    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-etc-openvswitch\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.140989 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140852    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-system-cni-dir\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.140989 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140878    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqqrc\" (UniqueName: \"kubernetes.io/projected/f5adf721-8d7d-4791-8410-575e3dfa6541-kube-api-access-bqqrc\") pod \"node-resolver-79mhk\" (UID: \"f5adf721-8d7d-4791-8410-575e3dfa6541\") " pod="openshift-dns/node-resolver-79mhk"
Sep 06 04:55:34.140989 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-var-lib-cni-multus\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.140989 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140904    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-system-cni-dir\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.140989 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140929    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-hostroot\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.140989 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140933    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-sysconfig\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.140989 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140956    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-system-cni-dir\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.140989 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140967    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-var-lib-kubelet\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.140994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqq4d\" (UniqueName: \"kubernetes.io/projected/23647459-2468-4daf-a8ba-b98d8ff206ca-kube-api-access-tqq4d\") pod \"iptables-alerter-4qwvq\" (UID: \"23647459-2468-4daf-a8ba-b98d8ff206ca\") " pod="openshift-network-operator/iptables-alerter-4qwvq"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141022    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-var-lib-openvswitch\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141065    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-run-ovn\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141127    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-cni-netd\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141180    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5adf721-8d7d-4791-8410-575e3dfa6541-tmp-dir\") pod \"node-resolver-79mhk\" (UID: \"f5adf721-8d7d-4791-8410-575e3dfa6541\") " pod="openshift-dns/node-resolver-79mhk"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141213    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-registration-dir\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141243    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99cc379d-e144-48d9-a9fc-309c11ecbc60-tmp\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141226    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141272    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-cni-bin\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-ovn-node-metrics-cert\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141354    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-ovnkube-script-lib\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-cnibin\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141392    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/837ae00e-1c4e-4e9c-8170-d7c018203862-cni-binary-copy\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141416    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141443    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-multus-cni-dir\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.141503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141452    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-cnibin\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141467    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-etc-selinux\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141490    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-slash\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141511    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-multus-cni-dir\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141517    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-run-systemd\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141596    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fccg\" (UniqueName: \"kubernetes.io/projected/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-kube-api-access-6fccg\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141638    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghrkf\" (UniqueName: \"kubernetes.io/projected/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-kube-api-access-ghrkf\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141663    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-var-lib-kubelet\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141722    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm9z4\" (UniqueName: \"kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4\") pod \"network-check-target-k5krn\" (UID: \"f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8\") " pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141748    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-sysctl-d\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141770    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-host\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141794    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-run-openvswitch\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141819    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-run-k8s-cni-cncf-io\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141871    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-run-multus-certs\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141895    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f5adf721-8d7d-4791-8410-575e3dfa6541-hosts-file\") pod \"node-resolver-79mhk\" (UID: \"f5adf721-8d7d-4791-8410-575e3dfa6541\") " pod="openshift-dns/node-resolver-79mhk"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141886    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-var-lib-kubelet\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141924    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed8f6484-9497-4655-853a-1b79956c4145-host\") pod \"node-ca-97hkw\" (UID: \"ed8f6484-9497-4655-853a-1b79956c4145\") " pod="openshift-image-registry/node-ca-97hkw"
Sep 06 04:55:34.142309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141944    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.141948    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-run-k8s-cni-cncf-io\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142003    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-run-multus-certs\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142025    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-systemd\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142098    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-cni-binary-copy\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142145    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h8fr\" (UniqueName: \"kubernetes.io/projected/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-kube-api-access-8h8fr\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142191    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-kubelet\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142225    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142257    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-multus-socket-dir-parent\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-run-netns\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142366    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-multus-socket-dir-parent\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142402    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-sysctl-conf\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142429    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6516247-fb60-4bc3-9db3-6c90ac893019-konnectivity-ca\") pod \"konnectivity-agent-4bhll\" (UID: \"f6516247-fb60-4bc3-9db3-6c90ac893019\") " pod="kube-system/konnectivity-agent-4bhll"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142466    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-socket-dir\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142492    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-cnibin\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142569    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-cni-binary-copy\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142620    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-kubernetes\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.143169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142628    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-run-netns\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142667    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xnlp\" (UniqueName: \"kubernetes.io/projected/99cc379d-e144-48d9-a9fc-309c11ecbc60-kube-api-access-7xnlp\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142692    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-cnibin\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-log-socket\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142721    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnrm4\" (UniqueName: \"kubernetes.io/projected/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-kube-api-access-wnrm4\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142760    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-multus-conf-dir\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142796    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed8f6484-9497-4655-853a-1b79956c4145-serviceca\") pod \"node-ca-97hkw\" (UID: \"ed8f6484-9497-4655-853a-1b79956c4145\") " pod="openshift-image-registry/node-ca-97hkw"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142826    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-run\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142843    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23647459-2468-4daf-a8ba-b98d8ff206ca-iptables-alerter-script\") pod \"iptables-alerter-4qwvq\" (UID: \"23647459-2468-4daf-a8ba-b98d8ff206ca\") " pod="openshift-network-operator/iptables-alerter-4qwvq"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142862    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-multus-conf-dir\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142885    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/837ae00e-1c4e-4e9c-8170-d7c018203862-multus-daemon-config\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.142960    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143067    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-lib-modules\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143108    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23647459-2468-4daf-a8ba-b98d8ff206ca-host-slash\") pod \"iptables-alerter-4qwvq\" (UID: \"23647459-2468-4daf-a8ba-b98d8ff206ca\") " pod="openshift-network-operator/iptables-alerter-4qwvq"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143183    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-modprobe-d\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143203    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-systemd-units\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.143821 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-node-log\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143234    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-env-overrides\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143255    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-device-dir\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-sys-fs\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143285    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d794e27a-146d-4d21-871c-3efaf67efeea-kubelet-config\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143302    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d794e27a-146d-4d21-871c-3efaf67efeea-dbus\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143317    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143331    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6516247-fb60-4bc3-9db3-6c90ac893019-agent-certs\") pod \"konnectivity-agent-4bhll\" (UID: \"f6516247-fb60-4bc3-9db3-6c90ac893019\") " pod="kube-system/konnectivity-agent-4bhll"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143330    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-var-lib-cni-bin\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-etc-kubernetes\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143378    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-sys\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143404    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-tuned\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143444    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-etc-kubernetes\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143448    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/837ae00e-1c4e-4e9c-8170-d7c018203862-multus-daemon-config\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.144371 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.143468    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/837ae00e-1c4e-4e9c-8170-d7c018203862-host-var-lib-cni-bin\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.155507 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.155480    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 06 04:55:34.158897 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.158875    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghrkf\" (UniqueName: \"kubernetes.io/projected/7143ad84-9eb1-4336-9ba4-48b6f73a8ee3-kube-api-access-ghrkf\") pod \"multus-additional-cni-plugins-jgkf2\" (UID: \"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3\") " pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.158979 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.158877    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zsnn\" (UniqueName: \"kubernetes.io/projected/837ae00e-1c4e-4e9c-8170-d7c018203862-kube-api-access-9zsnn\") pod \"multus-j6jvp\" (UID: \"837ae00e-1c4e-4e9c-8170-d7c018203862\") " pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.243784 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.243754    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h8fr\" (UniqueName: \"kubernetes.io/projected/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-kube-api-access-8h8fr\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.243940 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.243792    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-kubelet\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.243940 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.243819    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.243940 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.243848    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-sysctl-conf\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.243940 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.243875    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6516247-fb60-4bc3-9db3-6c90ac893019-konnectivity-ca\") pod \"konnectivity-agent-4bhll\" (UID: \"f6516247-fb60-4bc3-9db3-6c90ac893019\") " pod="kube-system/konnectivity-agent-4bhll"
Sep 06 04:55:34.243940 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.243893    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-kubelet\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.243940 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.243904    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.243940 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.243898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-socket-dir\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.243977    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-kubernetes\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244002    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-socket-dir\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244004    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xnlp\" (UniqueName: \"kubernetes.io/projected/99cc379d-e144-48d9-a9fc-309c11ecbc60-kube-api-access-7xnlp\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244031    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-sysctl-conf\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244066    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-log-socket\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244096    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnrm4\" (UniqueName: \"kubernetes.io/projected/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-kube-api-access-wnrm4\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244084    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-kubernetes\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244121    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed8f6484-9497-4655-853a-1b79956c4145-serviceca\") pod \"node-ca-97hkw\" (UID: \"ed8f6484-9497-4655-853a-1b79956c4145\") " pod="openshift-image-registry/node-ca-97hkw"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244146    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-run\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244148    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-log-socket\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244171    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23647459-2468-4daf-a8ba-b98d8ff206ca-iptables-alerter-script\") pod \"iptables-alerter-4qwvq\" (UID: \"23647459-2468-4daf-a8ba-b98d8ff206ca\") " pod="openshift-network-operator/iptables-alerter-4qwvq"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244218    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-lib-modules\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244257    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23647459-2468-4daf-a8ba-b98d8ff206ca-host-slash\") pod \"iptables-alerter-4qwvq\" (UID: \"23647459-2468-4daf-a8ba-b98d8ff206ca\") " pod="openshift-network-operator/iptables-alerter-4qwvq"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244276    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-run\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244279    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-modprobe-d\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.244328 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244330    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-systemd-units\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244356    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-node-log\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244385    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-modprobe-d\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244387    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-env-overrides\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244424    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-device-dir\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244453    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-sys-fs\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244441    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244480    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d794e27a-146d-4d21-871c-3efaf67efeea-kubelet-config\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244538    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-node-log\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244593    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23647459-2468-4daf-a8ba-b98d8ff206ca-host-slash\") pod \"iptables-alerter-4qwvq\" (UID: \"23647459-2468-4daf-a8ba-b98d8ff206ca\") " pod="openshift-network-operator/iptables-alerter-4qwvq"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244657    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6516247-fb60-4bc3-9db3-6c90ac893019-konnectivity-ca\") pod \"konnectivity-agent-4bhll\" (UID: \"f6516247-fb60-4bc3-9db3-6c90ac893019\") " pod="kube-system/konnectivity-agent-4bhll"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244765    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-lib-modules\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244821    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-device-dir\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-env-overrides\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244878    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-sys-fs\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244909    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-systemd-units\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.244959    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d794e27a-146d-4d21-871c-3efaf67efeea-dbus\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:34.245324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245013    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed8f6484-9497-4655-853a-1b79956c4145-serviceca\") pod \"node-ca-97hkw\" (UID: \"ed8f6484-9497-4655-853a-1b79956c4145\") " pod="openshift-image-registry/node-ca-97hkw"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245032    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d794e27a-146d-4d21-871c-3efaf67efeea-kubelet-config\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245060    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245078    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6516247-fb60-4bc3-9db3-6c90ac893019-agent-certs\") pod \"konnectivity-agent-4bhll\" (UID: \"f6516247-fb60-4bc3-9db3-6c90ac893019\") " pod="kube-system/konnectivity-agent-4bhll"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245100    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d794e27a-146d-4d21-871c-3efaf67efeea-dbus\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245131    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-sys\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-tuned\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245176    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245199    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtgxw\" (UniqueName: \"kubernetes.io/projected/ed8f6484-9497-4655-853a-1b79956c4145-kube-api-access-mtgxw\") pod \"node-ca-97hkw\" (UID: \"ed8f6484-9497-4655-853a-1b79956c4145\") " pod="openshift-image-registry/node-ca-97hkw"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245225    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245244    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-sys\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-ovnkube-config\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245291    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-run-netns\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245318    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-etc-openvswitch\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245344    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqqrc\" (UniqueName: \"kubernetes.io/projected/f5adf721-8d7d-4791-8410-575e3dfa6541-kube-api-access-bqqrc\") pod \"node-resolver-79mhk\" (UID: \"f5adf721-8d7d-4791-8410-575e3dfa6541\") " pod="openshift-dns/node-resolver-79mhk"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245369    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-sysconfig\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245394    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-var-lib-kubelet\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.246244 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245419    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqq4d\" (UniqueName: \"kubernetes.io/projected/23647459-2468-4daf-a8ba-b98d8ff206ca-kube-api-access-tqq4d\") pod \"iptables-alerter-4qwvq\" (UID: \"23647459-2468-4daf-a8ba-b98d8ff206ca\") " pod="openshift-network-operator/iptables-alerter-4qwvq"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245445    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-var-lib-openvswitch\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245470    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-run-ovn\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245495    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-cni-netd\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245520    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5adf721-8d7d-4791-8410-575e3dfa6541-tmp-dir\") pod \"node-resolver-79mhk\" (UID: \"f5adf721-8d7d-4791-8410-575e3dfa6541\") " pod="openshift-dns/node-resolver-79mhk"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-registration-dir\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245567    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99cc379d-e144-48d9-a9fc-309c11ecbc60-tmp\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245568    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23647459-2468-4daf-a8ba-b98d8ff206ca-iptables-alerter-script\") pod \"iptables-alerter-4qwvq\" (UID: \"23647459-2468-4daf-a8ba-b98d8ff206ca\") " pod="openshift-network-operator/iptables-alerter-4qwvq"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245590    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-cni-bin\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.245200    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245615    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-ovn-node-metrics-cert\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-ovnkube-script-lib\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.245683    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret podName:d794e27a-146d-4d21-871c-3efaf67efeea nodeName:}" failed. No retries permitted until 2026-09-06 04:55:34.745660399 +0000 UTC m=+2.115115648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret") pod "global-pull-secret-syncer-n2jlj" (UID: "d794e27a-146d-4d21-871c-3efaf67efeea") : object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-etc-selinux\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-slash\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245797    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-run-systemd\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245825    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fccg\" (UniqueName: \"kubernetes.io/projected/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-kube-api-access-6fccg\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:34.247056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245860    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-ovnkube-config\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.245288    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.245936    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs podName:d85cbcaa-00ae-4440-b7f0-7209f409a5cd nodeName:}" failed. No retries permitted until 2026-09-06 04:55:34.745922067 +0000 UTC m=+2.115377295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs") pod "network-metrics-daemon-dxn9s" (UID: "d85cbcaa-00ae-4440-b7f0-7209f409a5cd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246115    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-etc-selinux\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246161    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-slash\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:34.246166    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed8709ffad4ba828dceaf06e7817a74.slice/crio-a81646fe2d15a15db6b194af4610f28a6f5461da66c43b39dbcadb1fa9a3cd33 WatchSource:0}: Error finding container a81646fe2d15a15db6b194af4610f28a6f5461da66c43b39dbcadb1fa9a3cd33: Status 404 returned error can't find the container with id a81646fe2d15a15db6b194af4610f28a6f5461da66c43b39dbcadb1fa9a3cd33
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.245857    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm9z4\" (UniqueName: \"kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4\") pod \"network-check-target-k5krn\" (UID: \"f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8\") " pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246213    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-var-lib-openvswitch\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246235    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-sysctl-d\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246261    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-host\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246271    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-sysconfig\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246274    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-run-ovn\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-run-openvswitch\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246289    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-registration-dir\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246321    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-etc-openvswitch\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246327    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f5adf721-8d7d-4791-8410-575e3dfa6541-hosts-file\") pod \"node-resolver-79mhk\" (UID: \"f5adf721-8d7d-4791-8410-575e3dfa6541\") " pod="openshift-dns/node-resolver-79mhk"
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246331    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-var-lib-kubelet\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.247752 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246219    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-ovnkube-script-lib\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246359    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed8f6484-9497-4655-853a-1b79956c4145-host\") pod \"node-ca-97hkw\" (UID: \"ed8f6484-9497-4655-853a-1b79956c4145\") " pod="openshift-image-registry/node-ca-97hkw"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246384    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-systemd\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246463    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-systemd\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246501    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-cni-netd\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246521    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246203    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-run-systemd\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246735    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-sysctl-d\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246781    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99cc379d-e144-48d9-a9fc-309c11ecbc60-host\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246781    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-run-openvswitch\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246732    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-run-netns\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246830    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed8f6484-9497-4655-853a-1b79956c4145-host\") pod \"node-ca-97hkw\" (UID: \"ed8f6484-9497-4655-853a-1b79956c4145\") " pod="openshift-image-registry/node-ca-97hkw"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246858    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5adf721-8d7d-4791-8410-575e3dfa6541-tmp-dir\") pod \"node-resolver-79mhk\" (UID: \"f5adf721-8d7d-4791-8410-575e3dfa6541\") " pod="openshift-dns/node-resolver-79mhk"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.246949    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f5adf721-8d7d-4791-8410-575e3dfa6541-hosts-file\") pod \"node-resolver-79mhk\" (UID: \"f5adf721-8d7d-4791-8410-575e3dfa6541\") " pod="openshift-dns/node-resolver-79mhk"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.247043    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-host-cni-bin\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.248339    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/99cc379d-e144-48d9-a9fc-309c11ecbc60-etc-tuned\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.248585 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.248416    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6516247-fb60-4bc3-9db3-6c90ac893019-agent-certs\") pod \"konnectivity-agent-4bhll\" (UID: \"f6516247-fb60-4bc3-9db3-6c90ac893019\") " pod="kube-system/konnectivity-agent-4bhll"
Sep 06 04:55:34.249398 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.249271    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-ovn-node-metrics-cert\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.249439 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:34.249401    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a5564bf9fd0f70518b201a8b4b15358.slice/crio-aa2c8717fcecc5be8234916ea50b04c189fe4ebf38493412a6ed68aa17f136d4 WatchSource:0}: Error finding container aa2c8717fcecc5be8234916ea50b04c189fe4ebf38493412a6ed68aa17f136d4: Status 404 returned error can't find the container with id aa2c8717fcecc5be8234916ea50b04c189fe4ebf38493412a6ed68aa17f136d4
Sep 06 04:55:34.249710 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.249689    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99cc379d-e144-48d9-a9fc-309c11ecbc60-tmp\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.252551 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.252529    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 04:55:34.253232 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.253210    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 04:55:34.253397 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.253237    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 04:55:34.253397 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.253251    2578 projected.go:196] Error preparing data for projected volume kube-api-access-xm9z4 for pod openshift-network-diagnostics/network-check-target-k5krn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:34.253397 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.253312    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4 podName:f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:34.753290747 +0000 UTC m=+2.122745980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xm9z4" (UniqueName: "kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4") pod "network-check-target-k5krn" (UID: "f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:34.254643 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.254560    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xnlp\" (UniqueName: \"kubernetes.io/projected/99cc379d-e144-48d9-a9fc-309c11ecbc60-kube-api-access-7xnlp\") pod \"tuned-w895c\" (UID: \"99cc379d-e144-48d9-a9fc-309c11ecbc60\") " pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.254751 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.254644    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h8fr\" (UniqueName: \"kubernetes.io/projected/44fc7027-e8d7-4b44-8dc4-fb3b46c47b92-kube-api-access-8h8fr\") pod \"aws-ebs-csi-driver-node-wk7vj\" (UID: \"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.254751 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.254668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnrm4\" (UniqueName: \"kubernetes.io/projected/953a51c8-56ba-4c1c-9333-fb4cf193f4d6-kube-api-access-wnrm4\") pod \"ovnkube-node-zfz7w\" (UID: \"953a51c8-56ba-4c1c-9333-fb4cf193f4d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.256106 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.256080    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqq4d\" (UniqueName: \"kubernetes.io/projected/23647459-2468-4daf-a8ba-b98d8ff206ca-kube-api-access-tqq4d\") pod \"iptables-alerter-4qwvq\" (UID: \"23647459-2468-4daf-a8ba-b98d8ff206ca\") " pod="openshift-network-operator/iptables-alerter-4qwvq"
Sep 06 04:55:34.256318 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.256297    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtgxw\" (UniqueName: \"kubernetes.io/projected/ed8f6484-9497-4655-853a-1b79956c4145-kube-api-access-mtgxw\") pod \"node-ca-97hkw\" (UID: \"ed8f6484-9497-4655-853a-1b79956c4145\") " pod="openshift-image-registry/node-ca-97hkw"
Sep 06 04:55:34.256836 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.256815    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fccg\" (UniqueName: \"kubernetes.io/projected/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-kube-api-access-6fccg\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:34.256894 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.256835    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqqrc\" (UniqueName: \"kubernetes.io/projected/f5adf721-8d7d-4791-8410-575e3dfa6541-kube-api-access-bqqrc\") pod \"node-resolver-79mhk\" (UID: \"f5adf721-8d7d-4791-8410-575e3dfa6541\") " pod="openshift-dns/node-resolver-79mhk"
Sep 06 04:55:34.272082 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.272061    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 04:55:34.372916 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.372829    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jgkf2"
Sep 06 04:55:34.383508 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:34.383480    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7143ad84_9eb1_4336_9ba4_48b6f73a8ee3.slice/crio-99aeb18e066fdc0fe12083251ef5b52a20432e0252f0ed6913ff66d950cb54b6 WatchSource:0}: Error finding container 99aeb18e066fdc0fe12083251ef5b52a20432e0252f0ed6913ff66d950cb54b6: Status 404 returned error can't find the container with id 99aeb18e066fdc0fe12083251ef5b52a20432e0252f0ed6913ff66d950cb54b6
Sep 06 04:55:34.385325 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.385308    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j6jvp"
Sep 06 04:55:34.393546 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:34.393523    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod837ae00e_1c4e_4e9c_8170_d7c018203862.slice/crio-57016a1780b7df630803c33fb839c41adb9a4f1b07a900ce13e82afef9839dce WatchSource:0}: Error finding container 57016a1780b7df630803c33fb839c41adb9a4f1b07a900ce13e82afef9839dce: Status 404 returned error can't find the container with id 57016a1780b7df630803c33fb839c41adb9a4f1b07a900ce13e82afef9839dce
Sep 06 04:55:34.394671 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.394650    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w895c"
Sep 06 04:55:34.403130 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:34.403111    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99cc379d_e144_48d9_a9fc_309c11ecbc60.slice/crio-33ba93356bda6d765881dcaa16721bbea2affe4206824afe6ebd6b9985d248fa WatchSource:0}: Error finding container 33ba93356bda6d765881dcaa16721bbea2affe4206824afe6ebd6b9985d248fa: Status 404 returned error can't find the container with id 33ba93356bda6d765881dcaa16721bbea2affe4206824afe6ebd6b9985d248fa
Sep 06 04:55:34.407416 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.407400    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-79mhk"
Sep 06 04:55:34.416252 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:34.416235    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5adf721_8d7d_4791_8410_575e3dfa6541.slice/crio-2e0a99a44313e67ac5b16a0b44bcccc86704aebae2e9327f07cdfe1f84f6710e WatchSource:0}: Error finding container 2e0a99a44313e67ac5b16a0b44bcccc86704aebae2e9327f07cdfe1f84f6710e: Status 404 returned error can't find the container with id 2e0a99a44313e67ac5b16a0b44bcccc86704aebae2e9327f07cdfe1f84f6710e
Sep 06 04:55:34.426253 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.426232    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4qwvq"
Sep 06 04:55:34.434239 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:34.434219    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23647459_2468_4daf_a8ba_b98d8ff206ca.slice/crio-70f9d33e7fcf3876cd43c3df1718ba35b14b30ac896b6c97b5c7d174ff6f8103 WatchSource:0}: Error finding container 70f9d33e7fcf3876cd43c3df1718ba35b14b30ac896b6c97b5c7d174ff6f8103: Status 404 returned error can't find the container with id 70f9d33e7fcf3876cd43c3df1718ba35b14b30ac896b6c97b5c7d174ff6f8103
Sep 06 04:55:34.445211 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.445193    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:55:34.451412 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.451249    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4bhll"
Sep 06 04:55:34.453627 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:34.453604    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953a51c8_56ba_4c1c_9333_fb4cf193f4d6.slice/crio-90f6608b32cbb39176b2efe3dc1c622e0203b08e35d87db20075bfdbec063632 WatchSource:0}: Error finding container 90f6608b32cbb39176b2efe3dc1c622e0203b08e35d87db20075bfdbec063632: Status 404 returned error can't find the container with id 90f6608b32cbb39176b2efe3dc1c622e0203b08e35d87db20075bfdbec063632
Sep 06 04:55:34.459763 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:34.459743    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6516247_fb60_4bc3_9db3_6c90ac893019.slice/crio-b9f7de856d8fbd4e6ae6c5c1934775af3635f724012fcc8b756342078d9c59a4 WatchSource:0}: Error finding container b9f7de856d8fbd4e6ae6c5c1934775af3635f724012fcc8b756342078d9c59a4: Status 404 returned error can't find the container with id b9f7de856d8fbd4e6ae6c5c1934775af3635f724012fcc8b756342078d9c59a4
Sep 06 04:55:34.470558 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.470538    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj"
Sep 06 04:55:34.475598 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.475581    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-97hkw"
Sep 06 04:55:34.480061 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:34.480041    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44fc7027_e8d7_4b44_8dc4_fb3b46c47b92.slice/crio-3f112e8302dc39e8a406f9faa4f292cc935ad2436b810109aa5a60aada9ed563 WatchSource:0}: Error finding container 3f112e8302dc39e8a406f9faa4f292cc935ad2436b810109aa5a60aada9ed563: Status 404 returned error can't find the container with id 3f112e8302dc39e8a406f9faa4f292cc935ad2436b810109aa5a60aada9ed563
Sep 06 04:55:34.485912 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:55:34.485892    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8f6484_9497_4655_853a_1b79956c4145.slice/crio-47d9903156f452c6624a00102915587ba1e2887b5b9648e0d3ba8216fd4d896a WatchSource:0}: Error finding container 47d9903156f452c6624a00102915587ba1e2887b5b9648e0d3ba8216fd4d896a: Status 404 returned error can't find the container with id 47d9903156f452c6624a00102915587ba1e2887b5b9648e0d3ba8216fd4d896a
Sep 06 04:55:34.505956 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.505929    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 04:55:34.749382 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.749300    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:34.749382 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.749345    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:34.749610 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.749503    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:34.749610 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.749559    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs podName:d85cbcaa-00ae-4440-b7f0-7209f409a5cd nodeName:}" failed. No retries permitted until 2026-09-06 04:55:35.749541992 +0000 UTC m=+3.118997231 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs") pod "network-metrics-daemon-dxn9s" (UID: "d85cbcaa-00ae-4440-b7f0-7209f409a5cd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:34.749610 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.749503    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:34.749769 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.749655    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret podName:d794e27a-146d-4d21-871c-3efaf67efeea nodeName:}" failed. No retries permitted until 2026-09-06 04:55:35.749634088 +0000 UTC m=+3.119089316 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret") pod "global-pull-secret-syncer-n2jlj" (UID: "d794e27a-146d-4d21-871c-3efaf67efeea") : object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:34.850236 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:34.850200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm9z4\" (UniqueName: \"kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4\") pod \"network-check-target-k5krn\" (UID: \"f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8\") " pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:34.850456 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.850436    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 04:55:34.850532 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.850462    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 04:55:34.850532 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.850475    2578 projected.go:196] Error preparing data for projected volume kube-api-access-xm9z4 for pod openshift-network-diagnostics/network-check-target-k5krn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:34.850645 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:34.850533    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4 podName:f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:35.850513068 +0000 UTC m=+3.219968297 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xm9z4" (UniqueName: "kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4") pod "network-check-target-k5krn" (UID: "f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:35.089345 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.089219    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 04:50:34 +0000 UTC" deadline="2028-02-03 10:35:02.177990356 +0000 UTC"
Sep 06 04:55:35.089345 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.089295    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12365h39m27.088699816s"
Sep 06 04:55:35.165838 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.165723    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-97hkw" event={"ID":"ed8f6484-9497-4655-853a-1b79956c4145","Type":"ContainerStarted","Data":"47d9903156f452c6624a00102915587ba1e2887b5b9648e0d3ba8216fd4d896a"}
Sep 06 04:55:35.186766 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.186724    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj" event={"ID":"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92","Type":"ContainerStarted","Data":"3f112e8302dc39e8a406f9faa4f292cc935ad2436b810109aa5a60aada9ed563"}
Sep 06 04:55:35.208469 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.208429    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4bhll" event={"ID":"f6516247-fb60-4bc3-9db3-6c90ac893019","Type":"ContainerStarted","Data":"b9f7de856d8fbd4e6ae6c5c1934775af3635f724012fcc8b756342078d9c59a4"}
Sep 06 04:55:35.236589 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.236553    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4qwvq" event={"ID":"23647459-2468-4daf-a8ba-b98d8ff206ca","Type":"ContainerStarted","Data":"70f9d33e7fcf3876cd43c3df1718ba35b14b30ac896b6c97b5c7d174ff6f8103"}
Sep 06 04:55:35.251544 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.251511    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-79mhk" event={"ID":"f5adf721-8d7d-4791-8410-575e3dfa6541","Type":"ContainerStarted","Data":"2e0a99a44313e67ac5b16a0b44bcccc86704aebae2e9327f07cdfe1f84f6710e"}
Sep 06 04:55:35.287987 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.287951    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w895c" event={"ID":"99cc379d-e144-48d9-a9fc-309c11ecbc60","Type":"ContainerStarted","Data":"33ba93356bda6d765881dcaa16721bbea2affe4206824afe6ebd6b9985d248fa"}
Sep 06 04:55:35.303309 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.303251    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j6jvp" event={"ID":"837ae00e-1c4e-4e9c-8170-d7c018203862","Type":"ContainerStarted","Data":"57016a1780b7df630803c33fb839c41adb9a4f1b07a900ce13e82afef9839dce"}
Sep 06 04:55:35.323240 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.323198    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgkf2" event={"ID":"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3","Type":"ContainerStarted","Data":"99aeb18e066fdc0fe12083251ef5b52a20432e0252f0ed6913ff66d950cb54b6"}
Sep 06 04:55:35.342577 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.342494    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w" event={"ID":"953a51c8-56ba-4c1c-9333-fb4cf193f4d6","Type":"ContainerStarted","Data":"90f6608b32cbb39176b2efe3dc1c622e0203b08e35d87db20075bfdbec063632"}
Sep 06 04:55:35.381295 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.381257    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal" event={"ID":"6a5564bf9fd0f70518b201a8b4b15358","Type":"ContainerStarted","Data":"aa2c8717fcecc5be8234916ea50b04c189fe4ebf38493412a6ed68aa17f136d4"}
Sep 06 04:55:35.392591 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.392535    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-29.ec2.internal" event={"ID":"9ed8709ffad4ba828dceaf06e7817a74","Type":"ContainerStarted","Data":"a81646fe2d15a15db6b194af4610f28a6f5461da66c43b39dbcadb1fa9a3cd33"}
Sep 06 04:55:35.527981 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.527940    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 04:55:35.758801 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.758716    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:35.758801 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.758765    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:35.759037 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:35.758897    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:35.759037 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:35.758960    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs podName:d85cbcaa-00ae-4440-b7f0-7209f409a5cd nodeName:}" failed. No retries permitted until 2026-09-06 04:55:37.758942004 +0000 UTC m=+5.128397233 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs") pod "network-metrics-daemon-dxn9s" (UID: "d85cbcaa-00ae-4440-b7f0-7209f409a5cd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:35.759455 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:35.759432    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:35.759571 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:35.759490    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret podName:d794e27a-146d-4d21-871c-3efaf67efeea nodeName:}" failed. No retries permitted until 2026-09-06 04:55:37.759475261 +0000 UTC m=+5.128930486 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret") pod "global-pull-secret-syncer-n2jlj" (UID: "d794e27a-146d-4d21-871c-3efaf67efeea") : object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:35.859362 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:35.859306    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm9z4\" (UniqueName: \"kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4\") pod \"network-check-target-k5krn\" (UID: \"f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8\") " pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:35.859541 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:35.859525    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 04:55:35.859610 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:35.859551    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 04:55:35.859610 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:35.859565    2578 projected.go:196] Error preparing data for projected volume kube-api-access-xm9z4 for pod openshift-network-diagnostics/network-check-target-k5krn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:35.859750 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:35.859637    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4 podName:f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:37.859604389 +0000 UTC m=+5.229059618 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xm9z4" (UniqueName: "kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4") pod "network-check-target-k5krn" (UID: "f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:36.050798 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:36.050721    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 04:55:36.090256 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:36.090210    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 04:50:34 +0000 UTC" deadline="2028-04-16 13:18:00.851443491 +0000 UTC"
Sep 06 04:55:36.090256 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:36.090253    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14120h22m24.761194972s"
Sep 06 04:55:36.147795 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:36.146934    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:36.147795 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:36.147071    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:55:36.147795 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:36.147467    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:36.147795 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:36.147585    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:55:36.147795 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:36.147655    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:36.147795 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:36.147722    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:55:37.776953 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:37.776880    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:37.776953 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:37.776928    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:37.777451 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:37.777088    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:37.777451 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:37.777148    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs podName:d85cbcaa-00ae-4440-b7f0-7209f409a5cd nodeName:}" failed. No retries permitted until 2026-09-06 04:55:41.7771295 +0000 UTC m=+9.146584727 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs") pod "network-metrics-daemon-dxn9s" (UID: "d85cbcaa-00ae-4440-b7f0-7209f409a5cd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:37.777554 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:37.777501    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:37.777554 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:37.777546    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret podName:d794e27a-146d-4d21-871c-3efaf67efeea nodeName:}" failed. No retries permitted until 2026-09-06 04:55:41.777532832 +0000 UTC m=+9.146988059 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret") pod "global-pull-secret-syncer-n2jlj" (UID: "d794e27a-146d-4d21-871c-3efaf67efeea") : object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:37.877771 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:37.877736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm9z4\" (UniqueName: \"kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4\") pod \"network-check-target-k5krn\" (UID: \"f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8\") " pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:37.877970 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:37.877950    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 04:55:37.877970 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:37.877970    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 04:55:37.878159 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:37.877983    2578 projected.go:196] Error preparing data for projected volume kube-api-access-xm9z4 for pod openshift-network-diagnostics/network-check-target-k5krn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:37.878159 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:37.878050    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4 podName:f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:41.878032939 +0000 UTC m=+9.247488179 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xm9z4" (UniqueName: "kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4") pod "network-check-target-k5krn" (UID: "f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:38.147330 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:38.147250    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:38.147527 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:38.147378    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:55:38.147527 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:38.147441    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:38.147527 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:38.147511    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:55:38.147717 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:38.147567    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:38.147717 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:38.147645    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:55:40.147734 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:40.147694    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:40.148293 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:40.147827    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:55:40.148293 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:40.147942    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:40.148293 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:40.147960    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:40.148293 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:40.148088    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:55:40.148293 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:40.148180    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:55:41.812748 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:41.812551    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:41.812748 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:41.812603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:41.812748 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:41.812699    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:41.813478 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:41.812765    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs podName:d85cbcaa-00ae-4440-b7f0-7209f409a5cd nodeName:}" failed. No retries permitted until 2026-09-06 04:55:49.812747555 +0000 UTC m=+17.182202778 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs") pod "network-metrics-daemon-dxn9s" (UID: "d85cbcaa-00ae-4440-b7f0-7209f409a5cd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:41.813478 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:41.812699    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:41.813478 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:41.813134    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret podName:d794e27a-146d-4d21-871c-3efaf67efeea nodeName:}" failed. No retries permitted until 2026-09-06 04:55:49.813117158 +0000 UTC m=+17.182572394 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret") pod "global-pull-secret-syncer-n2jlj" (UID: "d794e27a-146d-4d21-871c-3efaf67efeea") : object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:41.913709 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:41.913032    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm9z4\" (UniqueName: \"kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4\") pod \"network-check-target-k5krn\" (UID: \"f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8\") " pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:41.913709 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:41.913198    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 04:55:41.913709 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:41.913220    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 04:55:41.913709 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:41.913232    2578 projected.go:196] Error preparing data for projected volume kube-api-access-xm9z4 for pod openshift-network-diagnostics/network-check-target-k5krn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:41.913709 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:41.913291    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4 podName:f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:49.913272191 +0000 UTC m=+17.282727419 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xm9z4" (UniqueName: "kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4") pod "network-check-target-k5krn" (UID: "f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:42.146880 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:42.146847    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:42.147058 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:42.146977    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:55:42.147389 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:42.147370    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:42.147482 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:42.147462    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:55:42.147554 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:42.147509    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:42.147606 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:42.147589    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:55:44.146993 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:44.146956    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:44.147397 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:44.147005    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:44.147397 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:44.147124    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:44.147397 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:44.147127    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:55:44.147397 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:44.147201    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:55:44.147397 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:44.147262    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:55:46.147622 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:46.147584    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:46.147622 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:46.147612    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:46.148076 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:46.147612    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:46.148076 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:46.147717    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:55:46.148076 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:46.147793    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:55:46.148076 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:46.147858    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:55:48.147939 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:48.147900    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:48.148370 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:48.147900    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:48.148370 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:48.148028    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:55:48.148370 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:48.148094    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:55:48.148370 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:48.147919    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:48.148370 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:48.148194    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:55:49.868404 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:49.868308    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:49.868404 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:49.868364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:49.868892 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:49.868487    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:49.868892 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:49.868551    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs podName:d85cbcaa-00ae-4440-b7f0-7209f409a5cd nodeName:}" failed. No retries permitted until 2026-09-06 04:56:05.868532839 +0000 UTC m=+33.237988064 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs") pod "network-metrics-daemon-dxn9s" (UID: "d85cbcaa-00ae-4440-b7f0-7209f409a5cd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:49.868892 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:49.868487    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:49.868892 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:49.868649    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret podName:d794e27a-146d-4d21-871c-3efaf67efeea nodeName:}" failed. No retries permitted until 2026-09-06 04:56:05.868629984 +0000 UTC m=+33.238085230 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret") pod "global-pull-secret-syncer-n2jlj" (UID: "d794e27a-146d-4d21-871c-3efaf67efeea") : object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:49.969285 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:49.969189    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm9z4\" (UniqueName: \"kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4\") pod \"network-check-target-k5krn\" (UID: \"f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8\") " pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:49.969449 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:49.969357    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 04:55:49.969449 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:49.969384    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 04:55:49.969449 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:49.969397    2578 projected.go:196] Error preparing data for projected volume kube-api-access-xm9z4 for pod openshift-network-diagnostics/network-check-target-k5krn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:49.969575 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:49.969456    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4 podName:f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8 nodeName:}" failed. No retries permitted until 2026-09-06 04:56:05.969436299 +0000 UTC m=+33.338891523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xm9z4" (UniqueName: "kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4") pod "network-check-target-k5krn" (UID: "f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:50.147031 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:50.146944    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:50.147189 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:50.147058    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:55:50.147189 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:50.146958    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:50.147189 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:50.147078    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:50.147189 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:50.147138    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:55:50.147350 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:50.147207    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:55:52.147796 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:52.147765    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:52.148339 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:52.147765    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:52.148339 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:52.147896    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:55:52.148339 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:52.147766    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:52.148339 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:52.147974    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:55:52.148339 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:52.148039    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:55:54.147424 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.147243    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:54.147881 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.147286    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:54.147881 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:54.147500    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:55:54.147881 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.147290    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:54.147881 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:54.147583    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:55:54.147881 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:54.147676    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:55:54.454763 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.454497    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-29.ec2.internal" event={"ID":"9ed8709ffad4ba828dceaf06e7817a74","Type":"ContainerStarted","Data":"d1e9e59c9453d75672d7ac3074e31247a5804c36dd198844100426ab3060dd1d"}
Sep 06 04:55:54.456063 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.456031    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-97hkw" event={"ID":"ed8f6484-9497-4655-853a-1b79956c4145","Type":"ContainerStarted","Data":"da046a2cf43ec95a3a8ec0d530b95c35c54c3535b3987210ce4addb64d28cdc9"}
Sep 06 04:55:54.457643 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.457587    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj" event={"ID":"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92","Type":"ContainerStarted","Data":"2951462fa60c80a277f9a920426635bd3fb155224bb89b20c64bbda75c01460e"}
Sep 06 04:55:54.458981 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.458960    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4bhll" event={"ID":"f6516247-fb60-4bc3-9db3-6c90ac893019","Type":"ContainerStarted","Data":"7b5d0d2c78cfd4d45d2362cd490c5100cce2daa375a615ac7b4c6f8c42a4339f"}
Sep 06 04:55:54.460401 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.460374    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-79mhk" event={"ID":"f5adf721-8d7d-4791-8410-575e3dfa6541","Type":"ContainerStarted","Data":"81d13078e2a2f305825fef02a6e8a2d54653fd96864771e930b3ae7bf49541b9"}
Sep 06 04:55:54.461670 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.461652    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w895c" event={"ID":"99cc379d-e144-48d9-a9fc-309c11ecbc60","Type":"ContainerStarted","Data":"5d4d8318815896c201a57e90324748a0eb494cbaec97c046bdd7acabafb20f65"}
Sep 06 04:55:54.463000 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.462982    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j6jvp" event={"ID":"837ae00e-1c4e-4e9c-8170-d7c018203862","Type":"ContainerStarted","Data":"fb6ed26bf69c0dabe6ee5ab25fb22f79dbce1c18e75e4d4137a591d08953ca73"}
Sep 06 04:55:54.464350 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.464328    2578 generic.go:356] "Generic (PLEG): container finished" podID="7143ad84-9eb1-4336-9ba4-48b6f73a8ee3" containerID="4929c35dd70d446854a293d5f0e6f798430575184bdc0ce75a2351a73c019b6c" exitCode=0
Sep 06 04:55:54.464432 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.464400    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgkf2" event={"ID":"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3","Type":"ContainerDied","Data":"4929c35dd70d446854a293d5f0e6f798430575184bdc0ce75a2351a73c019b6c"}
Sep 06 04:55:54.466433 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.466390    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-29.ec2.internal" podStartSLOduration=20.466374598 podStartE2EDuration="20.466374598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:55:54.466106063 +0000 UTC m=+21.835561309" watchObservedRunningTime="2026-09-06 04:55:54.466374598 +0000 UTC m=+21.835829844"
Sep 06 04:55:54.467678 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.467656    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w" event={"ID":"953a51c8-56ba-4c1c-9333-fb4cf193f4d6","Type":"ContainerStarted","Data":"15aaf2ef28da8eeac0552f1e17139b3a0cde7e421042475cbec9990b345e0b8d"}
Sep 06 04:55:54.467772 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.467713    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w" event={"ID":"953a51c8-56ba-4c1c-9333-fb4cf193f4d6","Type":"ContainerStarted","Data":"eb4b199b9d3dc5225444d9ad990496a30081059587bc2e2135c4e0541beaf985"}
Sep 06 04:55:54.467772 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.467747    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w" event={"ID":"953a51c8-56ba-4c1c-9333-fb4cf193f4d6","Type":"ContainerStarted","Data":"b388105982710219fc50ffa0758f9fc608f63fe1e640c18465376144f42432dd"}
Sep 06 04:55:54.467864 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.467778    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w" event={"ID":"953a51c8-56ba-4c1c-9333-fb4cf193f4d6","Type":"ContainerStarted","Data":"d7af5d8f218301e3483bcc78e016d2a7e4ace06f49a964e7d899f056d519e275"}
Sep 06 04:55:54.467864 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.467807    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w" event={"ID":"953a51c8-56ba-4c1c-9333-fb4cf193f4d6","Type":"ContainerStarted","Data":"307b74af6fdbfada68eb5222b781d76ad2f2fdb54350ffea870283676db02c5e"}
Sep 06 04:55:54.467864 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.467840    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w" event={"ID":"953a51c8-56ba-4c1c-9333-fb4cf193f4d6","Type":"ContainerStarted","Data":"93b4dcb85380e8c492c8c9fcb5cbd92eb909c0cc2d53fc4b667c4fffffe665a6"}
Sep 06 04:55:54.469090 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.469071    2578 generic.go:356] "Generic (PLEG): container finished" podID="6a5564bf9fd0f70518b201a8b4b15358" containerID="c6ba2ca5c18ac536f3d5c119c2eb5802296129ff622964c6ef240fa0c8aa8214" exitCode=0
Sep 06 04:55:54.469090 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.469097    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal" event={"ID":"6a5564bf9fd0f70518b201a8b4b15358","Type":"ContainerDied","Data":"c6ba2ca5c18ac536f3d5c119c2eb5802296129ff622964c6ef240fa0c8aa8214"}
Sep 06 04:55:54.520011 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.519966    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-97hkw" podStartSLOduration=2.360084035 podStartE2EDuration="21.519953149s" totalImagesPullingTime="19.159869114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:55:54.519775853 +0000 UTC m=+21.889231120" watchObservedRunningTime="2026-09-06 04:55:54.519953149 +0000 UTC m=+21.889408395"
Sep 06 04:55:54.520168 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.520104    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-j6jvp" podStartSLOduration=2.228749249 podStartE2EDuration="21.520099911s" totalImagesPullingTime="19.291350662s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:55:54.498865739 +0000 UTC m=+21.868320996" watchObservedRunningTime="2026-09-06 04:55:54.520099911 +0000 UTC m=+21.889555157"
Sep 06 04:55:54.553050 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.552990    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-w895c" podStartSLOduration=2.310112366 podStartE2EDuration="21.552977479s" totalImagesPullingTime="19.242865113s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:55:54.540127672 +0000 UTC m=+21.909582917" watchObservedRunningTime="2026-09-06 04:55:54.552977479 +0000 UTC m=+21.922432734"
Sep 06 04:55:54.553186 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.553103    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-79mhk" podStartSLOduration=2.325087682 podStartE2EDuration="21.553097263s" totalImagesPullingTime="19.228009581s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:55:54.552916372 +0000 UTC m=+21.922371618" watchObservedRunningTime="2026-09-06 04:55:54.553097263 +0000 UTC m=+21.922552509"
Sep 06 04:55:54.569703 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:54.569665    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4bhll" podStartSLOduration=2.3849346049999998 podStartE2EDuration="21.569653695s" totalImagesPullingTime="19.18471909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:55:54.569429733 +0000 UTC m=+21.938884980" watchObservedRunningTime="2026-09-06 04:55:54.569653695 +0000 UTC m=+21.939108941"
Sep 06 04:55:55.444703 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:55.444674    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 06 04:55:55.473844 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:55.473806    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj" event={"ID":"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92","Type":"ContainerStarted","Data":"a45106db0bb3ad952432bee1c0dab67b1ff3a96f8348cc9c592b13332736b092"}
Sep 06 04:55:55.475491 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:55.475458    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4qwvq" event={"ID":"23647459-2468-4daf-a8ba-b98d8ff206ca","Type":"ContainerStarted","Data":"88093be0eb80fa4c725234ca296d14017f7bf5a85f89719c974c0b335deac023"}
Sep 06 04:55:55.488285 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:55.488242    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4qwvq" podStartSLOduration=3.269755523 podStartE2EDuration="22.488224207s" totalImagesPullingTime="19.218468684s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:55:55.487986099 +0000 UTC m=+22.857441347" watchObservedRunningTime="2026-09-06 04:55:55.488224207 +0000 UTC m=+22.857679453"
Sep 06 04:55:56.101935 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:56.101828    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-06T04:55:55.444698035Z","UUID":"c6cca62a-d836-48ab-9710-374c0f064977","Handler":null,"Name":"","Endpoint":""}
Sep 06 04:55:56.104401 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:56.104032    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 06 04:55:56.104401 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:56.104069    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 06 04:55:56.147187 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:56.147153    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:56.147187 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:56.147184    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:56.147397 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:56.147285    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:55:56.147397 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:56.147349    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:55:56.147481 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:56.147416    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:56.147538 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:56.147511    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:55:56.480050 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:56.479933    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal" event={"ID":"6a5564bf9fd0f70518b201a8b4b15358","Type":"ContainerStarted","Data":"783c8975c4ce087e892660741fe143004591c3233430e060823958a5e4cad34f"}
Sep 06 04:55:56.493602 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:56.493559    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-29.ec2.internal" podStartSLOduration=22.493543745 podStartE2EDuration="22.493543745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:55:56.49317177 +0000 UTC m=+23.862627018" watchObservedRunningTime="2026-09-06 04:55:56.493543745 +0000 UTC m=+23.862998991"
Sep 06 04:55:57.256073 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:57.255761    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4bhll"
Sep 06 04:55:57.256938 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:57.256914    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4bhll"
Sep 06 04:55:57.487666 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:57.487626    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj" event={"ID":"44fc7027-e8d7-4b44-8dc4-fb3b46c47b92","Type":"ContainerStarted","Data":"6df39930fb30bfb2c95b80892b4076e6a583333e394d062ed15c77991bf194a1"}
Sep 06 04:55:57.491906 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:57.491873    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w" event={"ID":"953a51c8-56ba-4c1c-9333-fb4cf193f4d6","Type":"ContainerStarted","Data":"320473c6a385f9bdcd63a79ec13c5b7deb5efb5f96213671fafffdfe91cf4b44"}
Sep 06 04:55:57.492083 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:57.491965    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4bhll"
Sep 06 04:55:57.492514 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:57.492476    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4bhll"
Sep 06 04:55:57.502818 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:57.502771    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wk7vj" podStartSLOduration=2.856376727 podStartE2EDuration="24.502759455s" totalImagesPullingTime="21.646382728s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:55:57.502252306 +0000 UTC m=+24.871707553" watchObservedRunningTime="2026-09-06 04:55:57.502759455 +0000 UTC m=+24.872214700"
Sep 06 04:55:58.147756 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:58.147721    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:55:58.147756 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:58.147754    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:55:58.147997 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:58.147761    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:55:58.147997 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:58.147846    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:55:58.147997 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:58.147943    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:55:58.148186 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:55:58.148009    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:55:58.867629 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:58.867602    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:55:58.947736 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:58.947638    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:55:59.497769 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:59.497600    2578 generic.go:356] "Generic (PLEG): container finished" podID="7143ad84-9eb1-4336-9ba4-48b6f73a8ee3" containerID="f6338ed977d5f443704c36857005a274054c2d3ff97063625aa4b0a55eb2b8de" exitCode=0
Sep 06 04:55:59.497945 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:59.497657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgkf2" event={"ID":"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3","Type":"ContainerDied","Data":"f6338ed977d5f443704c36857005a274054c2d3ff97063625aa4b0a55eb2b8de"}
Sep 06 04:55:59.501833 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:55:59.501807    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w" event={"ID":"953a51c8-56ba-4c1c-9333-fb4cf193f4d6","Type":"ContainerStarted","Data":"beef1b589017416f66f00c2f38f11b5733e04175cac89f7042862b47ef6f21a5"}
Sep 06 04:56:00.147394 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.147360    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:56:00.147903 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.147397    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:56:00.147903 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.147360    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:56:00.147903 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:00.147491    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:56:00.147903 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:00.147588    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:56:00.147903 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:00.147665    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:56:00.504713 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.504679    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:56:00.504851 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.504787    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:56:00.504851 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.504843    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:56:00.520692 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.520668    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:56:00.521165 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.521144    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:56:00.544183 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.544099    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w" podStartSLOduration=8.311272628 podStartE2EDuration="27.54407853s" totalImagesPullingTime="19.232805902s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:55:59.53757667 +0000 UTC m=+26.907031909" watchObservedRunningTime="2026-09-06 04:56:00.54407853 +0000 UTC m=+27.913533777"
Sep 06 04:56:00.772905 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.772823    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:00.780376 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.780351    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n2jlj"]
Sep 06 04:56:00.780506 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.780432    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:56:00.780561 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:00.780517    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:56:00.783650 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.783610    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k5krn"]
Sep 06 04:56:00.783773 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.783720    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:56:00.783839 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:00.783816    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:56:00.784243 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.784222    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dxn9s"]
Sep 06 04:56:00.784359 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.784337    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:56:00.784461 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:00.784440    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:56:00.872512 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:00.872482    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:01.508207 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:01.508169    2578 generic.go:356] "Generic (PLEG): container finished" podID="7143ad84-9eb1-4336-9ba4-48b6f73a8ee3" containerID="aff618f003ef87a3543e2cf7d0497853f1bf7a02b5cd9b025b6eec5e1c6d041a" exitCode=0
Sep 06 04:56:01.508884 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:01.508259    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgkf2" event={"ID":"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3","Type":"ContainerDied","Data":"aff618f003ef87a3543e2cf7d0497853f1bf7a02b5cd9b025b6eec5e1c6d041a"}
Sep 06 04:56:01.748917 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:01.748880    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:01.801199 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:01.801124    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:02.147556 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:02.147522    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:56:02.147701 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:02.147538    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:56:02.147701 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:02.147659    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:56:02.147701 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:02.147695    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:56:03.148429 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:03.148185    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:56:03.148772 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:03.148535    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:56:03.515690 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:03.515597    2578 generic.go:356] "Generic (PLEG): container finished" podID="7143ad84-9eb1-4336-9ba4-48b6f73a8ee3" containerID="4e39e622e1bc952473a898aa19baec96289039e5109b6e5fe8c79f3a4a4805a9" exitCode=0
Sep 06 04:56:03.515690 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:03.515666    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgkf2" event={"ID":"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3","Type":"ContainerDied","Data":"4e39e622e1bc952473a898aa19baec96289039e5109b6e5fe8c79f3a4a4805a9"}
Sep 06 04:56:03.738851 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:03.738821    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:03.788318 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:03.788246    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:04.147208 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:04.147177    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:56:04.147391 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:04.147218    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:56:04.147391 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:04.147301    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:56:04.147490 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:04.147420    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:56:04.292325 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:04.292294    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:04.343074 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:04.343046    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:04.689875 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:04.689840    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:05.147836 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:05.147804    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:56:05.148010 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:05.147924    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:56:05.895088 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:05.895047    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:56:05.895623 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:05.895100    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:56:05.895623 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:05.895218    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:56:05.895623 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:05.895294    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs podName:d85cbcaa-00ae-4440-b7f0-7209f409a5cd nodeName:}" failed. No retries permitted until 2026-09-06 04:56:37.895273543 +0000 UTC m=+65.264728770 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs") pod "network-metrics-daemon-dxn9s" (UID: "d85cbcaa-00ae-4440-b7f0-7209f409a5cd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:56:05.895623 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:05.895220    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 04:56:05.895623 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:05.895383    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret podName:d794e27a-146d-4d21-871c-3efaf67efeea nodeName:}" failed. No retries permitted until 2026-09-06 04:56:37.89536374 +0000 UTC m=+65.264818976 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret") pod "global-pull-secret-syncer-n2jlj" (UID: "d794e27a-146d-4d21-871c-3efaf67efeea") : object "kube-system"/"original-pull-secret" not registered
Sep 06 04:56:05.996525 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:05.996489    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm9z4\" (UniqueName: \"kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4\") pod \"network-check-target-k5krn\" (UID: \"f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8\") " pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:56:05.996720 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:05.996653    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 04:56:05.996720 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:05.996679    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 04:56:05.996720 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:05.996694    2578 projected.go:196] Error preparing data for projected volume kube-api-access-xm9z4 for pod openshift-network-diagnostics/network-check-target-k5krn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:56:05.996848 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:05.996759    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4 podName:f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8 nodeName:}" failed. No retries permitted until 2026-09-06 04:56:37.996740404 +0000 UTC m=+65.366195629 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xm9z4" (UniqueName: "kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4") pod "network-check-target-k5krn" (UID: "f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:56:06.147422 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:06.147346    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:56:06.147566 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:06.147355    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:56:06.147566 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:06.147460    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5krn" podUID="f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8"
Sep 06 04:56:06.147566 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:06.147557    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n2jlj" podUID="d794e27a-146d-4d21-871c-3efaf67efeea"
Sep 06 04:56:07.147412 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.147381    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:56:07.147826 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:07.147514    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:56:07.459822 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.459755    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-29.ec2.internal" event="NodeReady"
Sep 06 04:56:07.459965 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.459904    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 06 04:56:07.497604 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.497570    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jzb7s"]
Sep 06 04:56:07.501355 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.501333    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mdzll"]
Sep 06 04:56:07.501473 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.501443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:07.503653 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.503625    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 06 04:56:07.503767 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.503630    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-prh2m\""
Sep 06 04:56:07.503874 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.503858    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 06 04:56:07.506815 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.506794    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mdzll"
Sep 06 04:56:07.509622 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.509269    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jzb7s"]
Sep 06 04:56:07.509622 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.509600    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 06 04:56:07.509743 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.509701    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7h5qb\""
Sep 06 04:56:07.509890 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.509869    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 06 04:56:07.509951 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.509926    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 06 04:56:07.515625 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.515603    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mdzll"]
Sep 06 04:56:07.604931 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.604905    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:07.607802 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.607773    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:07.607917 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.607815    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g29kr\" (UniqueName: \"kubernetes.io/projected/abb6c0e7-871b-4b97-9a23-906c01495e8a-kube-api-access-g29kr\") pod \"ingress-canary-mdzll\" (UID: \"abb6c0e7-871b-4b97-9a23-906c01495e8a\") " pod="openshift-ingress-canary/ingress-canary-mdzll"
Sep 06 04:56:07.607917 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.607844    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/379e032e-b73c-47bf-ad03-faa4ffc31425-config-volume\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:07.607917 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.607871    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/abb6c0e7-871b-4b97-9a23-906c01495e8a-cert\") pod \"ingress-canary-mdzll\" (UID: \"abb6c0e7-871b-4b97-9a23-906c01495e8a\") " pod="openshift-ingress-canary/ingress-canary-mdzll"
Sep 06 04:56:07.608062 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.607960    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df4wj\" (UniqueName: \"kubernetes.io/projected/379e032e-b73c-47bf-ad03-faa4ffc31425-kube-api-access-df4wj\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:07.608169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.608006    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/379e032e-b73c-47bf-ad03-faa4ffc31425-tmp-dir\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:07.655579 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.655554    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:07.707596 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.707558    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:07.708621 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.708603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:07.708665 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.708634    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g29kr\" (UniqueName: \"kubernetes.io/projected/abb6c0e7-871b-4b97-9a23-906c01495e8a-kube-api-access-g29kr\") pod \"ingress-canary-mdzll\" (UID: \"abb6c0e7-871b-4b97-9a23-906c01495e8a\") " pod="openshift-ingress-canary/ingress-canary-mdzll"
Sep 06 04:56:07.708665 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.708654    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/379e032e-b73c-47bf-ad03-faa4ffc31425-config-volume\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:07.708734 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.708677    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/abb6c0e7-871b-4b97-9a23-906c01495e8a-cert\") pod \"ingress-canary-mdzll\" (UID: \"abb6c0e7-871b-4b97-9a23-906c01495e8a\") " pod="openshift-ingress-canary/ingress-canary-mdzll"
Sep 06 04:56:07.708734 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.708728    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df4wj\" (UniqueName: \"kubernetes.io/projected/379e032e-b73c-47bf-ad03-faa4ffc31425-kube-api-access-df4wj\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:07.708817 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:07.708741    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:56:07.708817 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.708763    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/379e032e-b73c-47bf-ad03-faa4ffc31425-tmp-dir\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:07.708817 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:07.708799    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls podName:379e032e-b73c-47bf-ad03-faa4ffc31425 nodeName:}" failed. No retries permitted until 2026-09-06 04:56:08.208780074 +0000 UTC m=+35.578235303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls") pod "dns-default-jzb7s" (UID: "379e032e-b73c-47bf-ad03-faa4ffc31425") : secret "dns-default-metrics-tls" not found
Sep 06 04:56:07.709124 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.709104    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/379e032e-b73c-47bf-ad03-faa4ffc31425-tmp-dir\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:07.709249 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.709231    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/379e032e-b73c-47bf-ad03-faa4ffc31425-config-volume\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:07.712940 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.712882    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/abb6c0e7-871b-4b97-9a23-906c01495e8a-cert\") pod \"ingress-canary-mdzll\" (UID: \"abb6c0e7-871b-4b97-9a23-906c01495e8a\") " pod="openshift-ingress-canary/ingress-canary-mdzll"
Sep 06 04:56:07.716987 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.716964    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df4wj\" (UniqueName: \"kubernetes.io/projected/379e032e-b73c-47bf-ad03-faa4ffc31425-kube-api-access-df4wj\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:07.717155 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.717132    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g29kr\" (UniqueName: \"kubernetes.io/projected/abb6c0e7-871b-4b97-9a23-906c01495e8a-kube-api-access-g29kr\") pod \"ingress-canary-mdzll\" (UID: \"abb6c0e7-871b-4b97-9a23-906c01495e8a\") " pod="openshift-ingress-canary/ingress-canary-mdzll"
Sep 06 04:56:07.819867 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:07.819829    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mdzll"
Sep 06 04:56:08.093197 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:08.092956    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mdzll"]
Sep 06 04:56:08.097664 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:56:08.097643    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabb6c0e7_871b_4b97_9a23_906c01495e8a.slice/crio-546ef9213dbb26e1cb0d57f821d34f6d2c01693d3d7cb08211a22021e6a0987b WatchSource:0}: Error finding container 546ef9213dbb26e1cb0d57f821d34f6d2c01693d3d7cb08211a22021e6a0987b: Status 404 returned error can't find the container with id 546ef9213dbb26e1cb0d57f821d34f6d2c01693d3d7cb08211a22021e6a0987b
Sep 06 04:56:08.146945 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:08.146911    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:56:08.147116 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:08.146956    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:56:08.149807 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:08.149780    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 06 04:56:08.150196 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:08.149876    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 06 04:56:08.150196 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:08.149876    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dzjkx\""
Sep 06 04:56:08.150196 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:08.149934    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 06 04:56:08.213874 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:08.213835    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:08.214063 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:08.213948    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:56:08.214063 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:08.214003    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls podName:379e032e-b73c-47bf-ad03-faa4ffc31425 nodeName:}" failed. No retries permitted until 2026-09-06 04:56:09.213987413 +0000 UTC m=+36.583442640 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls") pod "dns-default-jzb7s" (UID: "379e032e-b73c-47bf-ad03-faa4ffc31425") : secret "dns-default-metrics-tls" not found
Sep 06 04:56:08.529134 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:08.529086    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mdzll" event={"ID":"abb6c0e7-871b-4b97-9a23-906c01495e8a","Type":"ContainerStarted","Data":"546ef9213dbb26e1cb0d57f821d34f6d2c01693d3d7cb08211a22021e6a0987b"}
Sep 06 04:56:08.533956 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:08.533931    2578 generic.go:356] "Generic (PLEG): container finished" podID="7143ad84-9eb1-4336-9ba4-48b6f73a8ee3" containerID="734da783c46b3a82e3fb65a00fee081afeda9c58d1a576db81241f965fd62b75" exitCode=0
Sep 06 04:56:08.534099 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:08.533978    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgkf2" event={"ID":"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3","Type":"ContainerDied","Data":"734da783c46b3a82e3fb65a00fee081afeda9c58d1a576db81241f965fd62b75"}
Sep 06 04:56:09.147159 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:09.147122    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:56:09.151919 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:09.150228    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b9kvk\""
Sep 06 04:56:09.151919 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:09.150341    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 06 04:56:09.220138 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:09.220102    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:09.220288 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:09.220265    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:56:09.220347 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:09.220336    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls podName:379e032e-b73c-47bf-ad03-faa4ffc31425 nodeName:}" failed. No retries permitted until 2026-09-06 04:56:11.220317425 +0000 UTC m=+38.589772669 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls") pod "dns-default-jzb7s" (UID: "379e032e-b73c-47bf-ad03-faa4ffc31425") : secret "dns-default-metrics-tls" not found
Sep 06 04:56:09.541166 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:09.541085    2578 generic.go:356] "Generic (PLEG): container finished" podID="7143ad84-9eb1-4336-9ba4-48b6f73a8ee3" containerID="a37d169bc35e38b2343e9cbc505966f09938f8af63caa46b2c5a6ae52ffbd8c5" exitCode=0
Sep 06 04:56:09.541302 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:09.541166    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgkf2" event={"ID":"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3","Type":"ContainerDied","Data":"a37d169bc35e38b2343e9cbc505966f09938f8af63caa46b2c5a6ae52ffbd8c5"}
Sep 06 04:56:10.547667 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:10.547632    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgkf2" event={"ID":"7143ad84-9eb1-4336-9ba4-48b6f73a8ee3","Type":"ContainerStarted","Data":"cb4ef67ff851a5745ef16435b6df491d60a314458f9d776306bc221bc3fb1925"}
Sep 06 04:56:10.549254 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:10.549229    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mdzll" event={"ID":"abb6c0e7-871b-4b97-9a23-906c01495e8a","Type":"ContainerStarted","Data":"f06a43f7c30ffc0ee25d6efdd3b373ab2d3499dd92d36ba93681dd7178a24495"}
Sep 06 04:56:10.569102 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:10.569057    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jgkf2" podStartSLOduration=7.937694705 podStartE2EDuration="37.569042262s" totalImagesPullingTime="29.631347557s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:10.567916262 +0000 UTC m=+37.937371535" watchObservedRunningTime="2026-09-06 04:56:10.569042262 +0000 UTC m=+37.938497508"
Sep 06 04:56:10.580923 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:10.580846    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mdzll" podStartSLOduration=1.327002479 podStartE2EDuration="3.580832405s" totalImagesPullingTime="2.253829926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:56:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:10.580342843 +0000 UTC m=+37.949798089" watchObservedRunningTime="2026-09-06 04:56:10.580832405 +0000 UTC m=+37.950287650"
Sep 06 04:56:10.693379 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:10.693205    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:10.793832 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:10.793799    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:10.856368 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:10.856303    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:11.235592 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:11.235551    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:11.235760 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:11.235708    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:56:11.235826 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:11.235775    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls podName:379e032e-b73c-47bf-ad03-faa4ffc31425 nodeName:}" failed. No retries permitted until 2026-09-06 04:56:15.235756662 +0000 UTC m=+42.605211886 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls") pod "dns-default-jzb7s" (UID: "379e032e-b73c-47bf-ad03-faa4ffc31425") : secret "dns-default-metrics-tls" not found
Sep 06 04:56:12.850616 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:12.850583    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:12.901699 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:12.901635    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:15.265842 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:15.265800    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:15.266208 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:15.265969    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:56:15.266208 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:15.266056    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls podName:379e032e-b73c-47bf-ad03-faa4ffc31425 nodeName:}" failed. No retries permitted until 2026-09-06 04:56:23.266032753 +0000 UTC m=+50.635487982 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls") pod "dns-default-jzb7s" (UID: "379e032e-b73c-47bf-ad03-faa4ffc31425") : secret "dns-default-metrics-tls" not found
Sep 06 04:56:15.828439 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:15.828404    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:15.924970 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:15.924933    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:23.321854 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:23.321810    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:23.322304 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:23.321949    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:56:23.322304 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:23.322032    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls podName:379e032e-b73c-47bf-ad03-faa4ffc31425 nodeName:}" failed. No retries permitted until 2026-09-06 04:56:39.322000502 +0000 UTC m=+66.691455725 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls") pod "dns-default-jzb7s" (UID: "379e032e-b73c-47bf-ad03-faa4ffc31425") : secret "dns-default-metrics-tls" not found
Sep 06 04:56:25.716203 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:25.716171    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:25.763385 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:25.763352    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:32.526145 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:32.526117    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfz7w"
Sep 06 04:56:32.670833 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:32.670792    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:32.720553 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:32.720527    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:33.760182 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:33.760149    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:37.926320 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:37.926288    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:56:37.926320 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:37.926326    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:56:37.928791 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:37.928742    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 06 04:56:37.928916 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:37.928900    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 06 04:56:37.937501 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:37.937479    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 06 04:56:37.937573 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:37.937536    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs podName:d85cbcaa-00ae-4440-b7f0-7209f409a5cd nodeName:}" failed. No retries permitted until 2026-09-06 04:57:41.937519747 +0000 UTC m=+129.306974975 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs") pod "network-metrics-daemon-dxn9s" (UID: "d85cbcaa-00ae-4440-b7f0-7209f409a5cd") : secret "metrics-daemon-secret" not found
Sep 06 04:56:37.940316 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:37.940296    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d794e27a-146d-4d21-871c-3efaf67efeea-original-pull-secret\") pod \"global-pull-secret-syncer-n2jlj\" (UID: \"d794e27a-146d-4d21-871c-3efaf67efeea\") " pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:56:38.026834 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:38.026789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm9z4\" (UniqueName: \"kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4\") pod \"network-check-target-k5krn\" (UID: \"f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8\") " pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:56:38.029760 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:38.029739    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 06 04:56:38.039545 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:38.039520    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 06 04:56:38.050558 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:38.050531    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm9z4\" (UniqueName: \"kubernetes.io/projected/f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8-kube-api-access-xm9z4\") pod \"network-check-target-k5krn\" (UID: \"f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8\") " pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:56:38.158875 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:38.158835    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dzjkx\""
Sep 06 04:56:38.161588 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:38.161568    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n2jlj"
Sep 06 04:56:38.166646 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:38.166619    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:56:38.441856 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:38.441784    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n2jlj"]
Sep 06 04:56:38.442408 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:38.442376    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k5krn"]
Sep 06 04:56:38.445036 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:56:38.444996    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd794e27a_146d_4d21_871c_3efaf67efeea.slice/crio-a33c6a3a8576d43ce2b8accbe233f8c3a832c6d2987fc6a6201f16f1f1901562 WatchSource:0}: Error finding container a33c6a3a8576d43ce2b8accbe233f8c3a832c6d2987fc6a6201f16f1f1901562: Status 404 returned error can't find the container with id a33c6a3a8576d43ce2b8accbe233f8c3a832c6d2987fc6a6201f16f1f1901562
Sep 06 04:56:38.445434 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:56:38.445410    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8bd34a6_1072_4097_ad3f_7b0f0a30b0e8.slice/crio-4edf6a38cf80001af448dd73fbd1e72a3bc3c97ea209824eea1623a63e1b57a4 WatchSource:0}: Error finding container 4edf6a38cf80001af448dd73fbd1e72a3bc3c97ea209824eea1623a63e1b57a4: Status 404 returned error can't find the container with id 4edf6a38cf80001af448dd73fbd1e72a3bc3c97ea209824eea1623a63e1b57a4
Sep 06 04:56:38.613315 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:38.613279    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k5krn" event={"ID":"f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8","Type":"ContainerStarted","Data":"4edf6a38cf80001af448dd73fbd1e72a3bc3c97ea209824eea1623a63e1b57a4"}
Sep 06 04:56:38.614394 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:38.614369    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n2jlj" event={"ID":"d794e27a-146d-4d21-871c-3efaf67efeea","Type":"ContainerStarted","Data":"a33c6a3a8576d43ce2b8accbe233f8c3a832c6d2987fc6a6201f16f1f1901562"}
Sep 06 04:56:38.779045 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:38.778950    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:38.835075 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:38.835043    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:56:39.337972 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:39.337941    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:56:39.338424 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:39.338100    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:56:39.338424 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:56:39.338165    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls podName:379e032e-b73c-47bf-ad03-faa4ffc31425 nodeName:}" failed. No retries permitted until 2026-09-06 04:57:11.338148888 +0000 UTC m=+98.707604126 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls") pod "dns-default-jzb7s" (UID: "379e032e-b73c-47bf-ad03-faa4ffc31425") : secret "dns-default-metrics-tls" not found
Sep 06 04:56:43.629291 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:43.629254    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k5krn" event={"ID":"f8bd34a6-1072-4097-ad3f-7b0f0a30b0e8","Type":"ContainerStarted","Data":"9b37676dfe5df7ee0ae31277f1830a14d9ecb6a7a3ecb8ca7e9fc418ae906ea7"}
Sep 06 04:56:43.629753 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:43.629373    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:56:43.630754 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:43.630732    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n2jlj" event={"ID":"d794e27a-146d-4d21-871c-3efaf67efeea","Type":"ContainerStarted","Data":"7b9a1b8292cbfba0b413bcc6be57545496538d36b476f4ecf9030768fb30410e"}
Sep 06 04:56:43.643592 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:43.643537    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k5krn" podStartSLOduration=66.044452926 podStartE2EDuration="1m10.643524809s" totalImagesPullingTime="4.599071883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:43.642220651 +0000 UTC m=+71.011675909" watchObservedRunningTime="2026-09-06 04:56:43.643524809 +0000 UTC m=+71.012980055"
Sep 06 04:56:43.654552 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:43.654508    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n2jlj" podStartSLOduration=66.050618662 podStartE2EDuration="1m10.654494549s" totalImagesPullingTime="4.603875887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:43.654065681 +0000 UTC m=+71.023520926" watchObservedRunningTime="2026-09-06 04:56:43.654494549 +0000 UTC m=+71.023949796"
Sep 06 04:56:59.794868 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.794838    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt"]
Sep 06 04:56:59.797886 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.797865    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt"
Sep 06 04:56:59.800997 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.800969    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 06 04:56:59.800997 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.800975    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 06 04:56:59.801175 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.801058    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 06 04:56:59.801175 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.801094    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 06 04:56:59.801334 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.801313    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-tz7xz\""
Sep 06 04:56:59.805093 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.805074    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt"]
Sep 06 04:56:59.810706 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.810687    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"]
Sep 06 04:56:59.813477 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.813464    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.815641 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.815624    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 06 04:56:59.815730 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.815711    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 06 04:56:59.815730 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.815719    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 06 04:56:59.815826 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.815711    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 06 04:56:59.825063 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.825043    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"]
Sep 06 04:56:59.879548 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.879514    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-hub\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.879548 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.879549    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ef26a853-6002-4886-8fd8-3a56ce105565-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78649c677d-6m6nt\" (UID: \"ef26a853-6002-4886-8fd8-3a56ce105565\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt"
Sep 06 04:56:59.879721 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.879567    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.879721 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.879637    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.879721 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.879666    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.879721 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.879692    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82tjs\" (UniqueName: \"kubernetes.io/projected/ef26a853-6002-4886-8fd8-3a56ce105565-kube-api-access-82tjs\") pod \"managed-serviceaccount-addon-agent-78649c677d-6m6nt\" (UID: \"ef26a853-6002-4886-8fd8-3a56ce105565\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt"
Sep 06 04:56:59.879721 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.879712    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-ca\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.879873 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.879734    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5lvc\" (UniqueName: \"kubernetes.io/projected/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-kube-api-access-p5lvc\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.980951 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.980916    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82tjs\" (UniqueName: \"kubernetes.io/projected/ef26a853-6002-4886-8fd8-3a56ce105565-kube-api-access-82tjs\") pod \"managed-serviceaccount-addon-agent-78649c677d-6m6nt\" (UID: \"ef26a853-6002-4886-8fd8-3a56ce105565\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt"
Sep 06 04:56:59.981096 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.980953    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-ca\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.981096 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.980978    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5lvc\" (UniqueName: \"kubernetes.io/projected/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-kube-api-access-p5lvc\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.981096 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.981034    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-hub\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.981096 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.981057    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ef26a853-6002-4886-8fd8-3a56ce105565-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78649c677d-6m6nt\" (UID: \"ef26a853-6002-4886-8fd8-3a56ce105565\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt"
Sep 06 04:56:59.981096 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.981071    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.981343 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.981102    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.981343 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.981135    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.981869 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.981839    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.983561 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.983539    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-ca\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.984123 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.984087    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.984220 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.984195    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.984270 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.984212    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ef26a853-6002-4886-8fd8-3a56ce105565-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78649c677d-6m6nt\" (UID: \"ef26a853-6002-4886-8fd8-3a56ce105565\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt"
Sep 06 04:56:59.984270 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.984231    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-hub\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.987694 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.987671    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5lvc\" (UniqueName: \"kubernetes.io/projected/bf76b39e-fe5f-4d1d-9e0e-6177bf241302-kube-api-access-p5lvc\") pod \"cluster-proxy-proxy-agent-5cbf55c7c8-n24fh\" (UID: \"bf76b39e-fe5f-4d1d-9e0e-6177bf241302\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:56:59.987977 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:56:59.987954    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82tjs\" (UniqueName: \"kubernetes.io/projected/ef26a853-6002-4886-8fd8-3a56ce105565-kube-api-access-82tjs\") pod \"managed-serviceaccount-addon-agent-78649c677d-6m6nt\" (UID: \"ef26a853-6002-4886-8fd8-3a56ce105565\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt"
Sep 06 04:57:00.112946 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:00.112865    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt"
Sep 06 04:57:00.121139 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:00.121113    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 04:57:00.380782 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:00.380753    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt"]
Sep 06 04:57:00.383428 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:57:00.383398    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef26a853_6002_4886_8fd8_3a56ce105565.slice/crio-01bff697cc04ba5d7e4f19f7d9735a6ebb73834369111a03de6a15a1b5a14036 WatchSource:0}: Error finding container 01bff697cc04ba5d7e4f19f7d9735a6ebb73834369111a03de6a15a1b5a14036: Status 404 returned error can't find the container with id 01bff697cc04ba5d7e4f19f7d9735a6ebb73834369111a03de6a15a1b5a14036
Sep 06 04:57:00.388461 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:00.388394    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"]
Sep 06 04:57:00.391125 ip-10-0-134-29 kubenswrapper[2578]: W0906 04:57:00.391105    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf76b39e_fe5f_4d1d_9e0e_6177bf241302.slice/crio-881bf0dac5ec089b9570ac9707cb3a207529d0c63c6a3fe5a28315146acf4767 WatchSource:0}: Error finding container 881bf0dac5ec089b9570ac9707cb3a207529d0c63c6a3fe5a28315146acf4767: Status 404 returned error can't find the container with id 881bf0dac5ec089b9570ac9707cb3a207529d0c63c6a3fe5a28315146acf4767
Sep 06 04:57:00.678168 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:00.678047    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt" event={"ID":"ef26a853-6002-4886-8fd8-3a56ce105565","Type":"ContainerStarted","Data":"01bff697cc04ba5d7e4f19f7d9735a6ebb73834369111a03de6a15a1b5a14036"}
Sep 06 04:57:00.680706 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:00.680676    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh" event={"ID":"bf76b39e-fe5f-4d1d-9e0e-6177bf241302","Type":"ContainerStarted","Data":"881bf0dac5ec089b9570ac9707cb3a207529d0c63c6a3fe5a28315146acf4767"}
Sep 06 04:57:03.692912 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:03.692839    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt" event={"ID":"ef26a853-6002-4886-8fd8-3a56ce105565","Type":"ContainerStarted","Data":"b9e62d00a1c7a1bba0c5dc27bee4002725b864fafda730fb003293ce53297c1a"}
Sep 06 04:57:03.707409 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:03.707357    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt" podStartSLOduration=1.959836132 podStartE2EDuration="4.707340299s" totalImagesPullingTime="2.747504167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:57:03.706548315 +0000 UTC m=+91.076003563" watchObservedRunningTime="2026-09-06 04:57:03.707340299 +0000 UTC m=+91.076795545"
Sep 06 04:57:04.697918 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:04.697892    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh" event={"ID":"bf76b39e-fe5f-4d1d-9e0e-6177bf241302","Type":"ContainerStarted","Data":"918742e98de92115337c60c790d3d32ebda5ffed0d66d8be37525ec65f94613c"}
Sep 06 04:57:04.698182 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:04.697938    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh" event={"ID":"bf76b39e-fe5f-4d1d-9e0e-6177bf241302","Type":"ContainerStarted","Data":"5cb12621a37cce7f8c4ce71c161538d2751f521d92297fdd755a4bc432c73998"}
Sep 06 04:57:05.705976 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:05.705938    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh" event={"ID":"bf76b39e-fe5f-4d1d-9e0e-6177bf241302","Type":"ContainerStarted","Data":"6e4885bf0c089a9451304430d83a7ab9d1a847a3709a22bf4cd41529bef4ad97"}
Sep 06 04:57:05.722898 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:05.722849    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh" podStartSLOduration=2.573808905 podStartE2EDuration="6.722836408s" totalImagesPullingTime="4.149027503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:57:05.721690346 +0000 UTC m=+93.091145594" watchObservedRunningTime="2026-09-06 04:57:05.722836408 +0000 UTC m=+93.092291655"
Sep 06 04:57:11.365569 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:11.365532    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:57:11.365937 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:57:11.365665    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:57:11.365937 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:57:11.365724    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls podName:379e032e-b73c-47bf-ad03-faa4ffc31425 nodeName:}" failed. No retries permitted until 2026-09-06 04:58:15.365705625 +0000 UTC m=+162.735160855 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls") pod "dns-default-jzb7s" (UID: "379e032e-b73c-47bf-ad03-faa4ffc31425") : secret "dns-default-metrics-tls" not found
Sep 06 04:57:14.638668 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:14.638638    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k5krn"
Sep 06 04:57:41.976147 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:57:41.976091    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:57:41.976633 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:57:41.976256    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 06 04:57:41.976633 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:57:41.976343    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs podName:d85cbcaa-00ae-4440-b7f0-7209f409a5cd nodeName:}" failed. No retries permitted until 2026-09-06 04:59:43.976324575 +0000 UTC m=+251.345779813 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs") pod "network-metrics-daemon-dxn9s" (UID: "d85cbcaa-00ae-4440-b7f0-7209f409a5cd") : secret "metrics-daemon-secret" not found
Sep 06 04:58:10.514464 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:58:10.514422    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jzb7s" podUID="379e032e-b73c-47bf-ad03-faa4ffc31425"
Sep 06 04:58:10.892376 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:58:10.892350    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:58:12.159849 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:58:12.159811    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 04:58:15.407622 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:58:15.407581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 04:58:15.407993 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:58:15.407711    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:58:15.407993 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:58:15.407767    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls podName:379e032e-b73c-47bf-ad03-faa4ffc31425 nodeName:}" failed. No retries permitted until 2026-09-06 05:00:17.407751175 +0000 UTC m=+284.777206399 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls") pod "dns-default-jzb7s" (UID: "379e032e-b73c-47bf-ad03-faa4ffc31425") : secret "dns-default-metrics-tls" not found
Sep 06 04:58:27.148059 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:58:27.147951    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:58:50.918773 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:58:50.918737    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:04.039140 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:04.039107    2578 generic.go:356] "Generic (PLEG): container finished" podID="ef26a853-6002-4886-8fd8-3a56ce105565" containerID="b9e62d00a1c7a1bba0c5dc27bee4002725b864fafda730fb003293ce53297c1a" exitCode=255
Sep 06 04:59:04.039604 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:04.039152    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt" event={"ID":"ef26a853-6002-4886-8fd8-3a56ce105565","Type":"ContainerDied","Data":"b9e62d00a1c7a1bba0c5dc27bee4002725b864fafda730fb003293ce53297c1a"}
Sep 06 04:59:04.039604 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:04.039418    2578 scope.go:117] "RemoveContainer" containerID="b9e62d00a1c7a1bba0c5dc27bee4002725b864fafda730fb003293ce53297c1a"
Sep 06 04:59:05.044184 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:05.044146    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78649c677d-6m6nt" event={"ID":"ef26a853-6002-4886-8fd8-3a56ce105565","Type":"ContainerStarted","Data":"0d2b8e978009b66dbc3a5e4120eeb844ed6dace7c04b26ef24f7f56b8a756e8c"}
Sep 06 04:59:14.824081 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:14.824045    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:41.608446 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:41.608384    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:42.729073 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:42.729038    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:43.992835 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:43.992797    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 04:59:43.993213 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:43.992941    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 06 04:59:43.993213 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:43.993004    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs podName:d85cbcaa-00ae-4440-b7f0-7209f409a5cd nodeName:}" failed. No retries permitted until 2026-09-06 05:01:45.992988271 +0000 UTC m=+373.362443495 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs") pod "network-metrics-daemon-dxn9s" (UID: "d85cbcaa-00ae-4440-b7f0-7209f409a5cd") : secret "metrics-daemon-secret" not found
Sep 06 04:59:44.308280 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:44.308196    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:45.180251 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.180219    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:45.253306 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.253264    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:45.427865 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.427834    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:45.526291 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.526206    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:45.585164 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.582431    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:45.765099 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.765067    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vkvc2"]
Sep 06 04:59:45.767965 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.767940    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:45.770315 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.770296    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 06 04:59:45.770423 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.770333    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 06 04:59:45.771324 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.771303    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 06 04:59:45.771428 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.771303    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qr8nl\""
Sep 06 04:59:45.771428 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.771346    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 06 04:59:45.784483 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.784430    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vkvc2"]
Sep 06 04:59:45.790343 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.790323    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:45.860597 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.860566    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:45.907564 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.907536    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:45.907692 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.907573    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:45.907692 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.907637    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qqvq\" (UniqueName: \"kubernetes.io/projected/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-api-access-6qqvq\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:45.907692 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.907686    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d78dc73-4a38-43c7-865a-c74aeca7478f-data-volume\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:45.907817 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.907741    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d78dc73-4a38-43c7-865a-c74aeca7478f-crio-socket\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:45.950356 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:45.950327    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:46.008056 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:46.008007    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d78dc73-4a38-43c7-865a-c74aeca7478f-data-volume\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:46.008182 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:46.008085    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d78dc73-4a38-43c7-865a-c74aeca7478f-crio-socket\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:46.008182 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:46.008113    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:46.008182 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:46.008133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:46.008182 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:46.008150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qqvq\" (UniqueName: \"kubernetes.io/projected/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-api-access-6qqvq\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:46.008372 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:46.008210    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 04:59:46.008372 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:46.008238    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d78dc73-4a38-43c7-865a-c74aeca7478f-crio-socket\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:46.008372 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:46.008269    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:46.008372 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:46.008306    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm podName:4d78dc73-4a38-43c7-865a-c74aeca7478f nodeName:}" failed. No retries permitted until 2026-09-06 04:59:46.508282559 +0000 UTC m=+253.877737798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vkvc2" (UID: "4d78dc73-4a38-43c7-865a-c74aeca7478f") : configmap "kube-rbac-proxy" not found
Sep 06 04:59:46.008372 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:46.008326    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls podName:4d78dc73-4a38-43c7-865a-c74aeca7478f nodeName:}" failed. No retries permitted until 2026-09-06 04:59:46.508316693 +0000 UTC m=+253.877771922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vkvc2" (UID: "4d78dc73-4a38-43c7-865a-c74aeca7478f") : secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:46.008372 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:46.008362    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d78dc73-4a38-43c7-865a-c74aeca7478f-data-volume\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:46.018760 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:46.018733    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qqvq\" (UniqueName: \"kubernetes.io/projected/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-api-access-6qqvq\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:46.475955 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:46.475923    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:46.512596 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:46.512564    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:46.512690 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:46.512602    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:46.512690 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:46.512684    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 04:59:46.512759 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:46.512713    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:46.512759 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:46.512752    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm podName:4d78dc73-4a38-43c7-865a-c74aeca7478f nodeName:}" failed. No retries permitted until 2026-09-06 04:59:47.512733286 +0000 UTC m=+254.882188545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vkvc2" (UID: "4d78dc73-4a38-43c7-865a-c74aeca7478f") : configmap "kube-rbac-proxy" not found
Sep 06 04:59:46.512826 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:46.512767    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls podName:4d78dc73-4a38-43c7-865a-c74aeca7478f nodeName:}" failed. No retries permitted until 2026-09-06 04:59:47.512760312 +0000 UTC m=+254.882215536 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vkvc2" (UID: "4d78dc73-4a38-43c7-865a-c74aeca7478f") : secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:46.577721 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:46.577686    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:47.517782 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:47.517723    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:47.518728 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:47.518708    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:47.518775 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:47.518739    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:47.518846 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:47.518835    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:47.518888 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:47.518880    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls podName:4d78dc73-4a38-43c7-865a-c74aeca7478f nodeName:}" failed. No retries permitted until 2026-09-06 04:59:49.518868981 +0000 UTC m=+256.888324205 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vkvc2" (UID: "4d78dc73-4a38-43c7-865a-c74aeca7478f") : secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:47.518934 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:47.518915    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 04:59:47.518968 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:47.518961    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm podName:4d78dc73-4a38-43c7-865a-c74aeca7478f nodeName:}" failed. No retries permitted until 2026-09-06 04:59:49.51894976 +0000 UTC m=+256.888404984 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vkvc2" (UID: "4d78dc73-4a38-43c7-865a-c74aeca7478f") : configmap "kube-rbac-proxy" not found
Sep 06 04:59:47.530726 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:47.530702    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-79mhk_f5adf721-8d7d-4791-8410-575e3dfa6541/dns-node-resolver/0.log"
Sep 06 04:59:47.570042 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:47.569998    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:48.335143 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:48.335108    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:48.531548 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:48.531517    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-97hkw_ed8f6484-9497-4655-853a-1b79956c4145/node-ca/0.log"
Sep 06 04:59:49.487981 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:49.487945    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:49.533434 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:49.533398    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:49.533826 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:49.533496    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:49.533826 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:49.533549    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:49.533826 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:49.533575    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 04:59:49.533826 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:49.533617    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls podName:4d78dc73-4a38-43c7-865a-c74aeca7478f nodeName:}" failed. No retries permitted until 2026-09-06 04:59:53.533599572 +0000 UTC m=+260.903054812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vkvc2" (UID: "4d78dc73-4a38-43c7-865a-c74aeca7478f") : secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:49.533826 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:49.533632    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm podName:4d78dc73-4a38-43c7-865a-c74aeca7478f nodeName:}" failed. No retries permitted until 2026-09-06 04:59:53.533625041 +0000 UTC m=+260.903080265 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vkvc2" (UID: "4d78dc73-4a38-43c7-865a-c74aeca7478f") : configmap "kube-rbac-proxy" not found
Sep 06 04:59:49.589139 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:49.589107    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:49.732135 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:49.732112    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mdzll_abb6c0e7-871b-4b97-9a23-906c01495e8a/serve-healthcheck-canary/0.log"
Sep 06 04:59:49.763455 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:49.763385    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:49.865611 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:49.865578    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:49.998543 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:49.998517    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:50.056899 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:50.056830    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:50.182845 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:50.182806    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:51.182945 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:51.182910    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:51.333881 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:51.333853    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:51.586964 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:51.586884    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:51.794837 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:51.794805    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:51.985084 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:51.985048    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:52.382835 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:52.382800    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:52.794847 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:52.794765    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:53.178236 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:53.178204    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:53.378932 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:53.378899    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:53.428337 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:53.428265    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:53.562136 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:53.562096    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:53.562136 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:53.562140    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 04:59:53.562357 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:53.562200    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 04:59:53.562357 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:53.562266    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:53.562357 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:53.562287    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm podName:4d78dc73-4a38-43c7-865a-c74aeca7478f nodeName:}" failed. No retries permitted until 2026-09-06 05:00:01.562270173 +0000 UTC m=+268.931725417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vkvc2" (UID: "4d78dc73-4a38-43c7-865a-c74aeca7478f") : configmap "kube-rbac-proxy" not found
Sep 06 04:59:53.562357 ip-10-0-134-29 kubenswrapper[2578]: E0906 04:59:53.562315    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls podName:4d78dc73-4a38-43c7-865a-c74aeca7478f nodeName:}" failed. No retries permitted until 2026-09-06 05:00:01.562303033 +0000 UTC m=+268.931758256 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vkvc2" (UID: "4d78dc73-4a38-43c7-865a-c74aeca7478f") : secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:53.785135 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:53.785062    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:53.991249 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:53.991214    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:54.380702 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:54.380671    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:54.636857 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:54.636774    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:54.780707 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:54.780671    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:55.380228 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:55.380194    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:55.822826 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:55.822793    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:55.923800 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:55.923766    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:56.181131 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:56.181099    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:56.385503 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:56.385473    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:56.784296 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:56.784265    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:57.382749 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:57.382720    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:57.980986 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:57.980956    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:58.184440 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:58.184400    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:58.380913 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:58.380882    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:58.638169 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:58.638081    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:58.778938 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:58.778906    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:59.386108 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:59.386077    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:59.488152 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:59.488117    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:59.633531 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:59.633503    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:59.782929 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:59.782850    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 04:59:59.984143 ip-10-0-134-29 kubenswrapper[2578]: I0906 04:59:59.984110    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:00.180288 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:00.180258    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:00.582675 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:00.582592    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:00.847414 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:00.847334    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:01.627025 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:01.626982    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 05:00:01.627025 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:01.627028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 05:00:01.627484 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:01.627467    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d78dc73-4a38-43c7-865a-c74aeca7478f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 05:00:01.629172 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:01.629155    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d78dc73-4a38-43c7-865a-c74aeca7478f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vkvc2\" (UID: \"4d78dc73-4a38-43c7-865a-c74aeca7478f\") " pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 05:00:01.676411 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:01.676380    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vkvc2"
Sep 06 05:00:01.946426 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:01.946355    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vkvc2"]
Sep 06 05:00:01.949167 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:00:01.949134    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d78dc73_4a38_43c7_865a_c74aeca7478f.slice/crio-7a0ea9755cd60759286293b98daf1f622649eef0fc9e6b51be9fb077bf7fe529 WatchSource:0}: Error finding container 7a0ea9755cd60759286293b98daf1f622649eef0fc9e6b51be9fb077bf7fe529: Status 404 returned error can't find the container with id 7a0ea9755cd60759286293b98daf1f622649eef0fc9e6b51be9fb077bf7fe529
Sep 06 05:00:02.202289 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:02.202210    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vkvc2" event={"ID":"4d78dc73-4a38-43c7-865a-c74aeca7478f","Type":"ContainerStarted","Data":"45d34d9bc97e5d98c826fa1891dc5bdb35c324fccf9090f808b04945ed82d99f"}
Sep 06 05:00:02.202289 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:02.202262    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vkvc2" event={"ID":"4d78dc73-4a38-43c7-865a-c74aeca7478f","Type":"ContainerStarted","Data":"7a0ea9755cd60759286293b98daf1f622649eef0fc9e6b51be9fb077bf7fe529"}
Sep 06 05:00:04.209270 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:04.209237    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vkvc2" event={"ID":"4d78dc73-4a38-43c7-865a-c74aeca7478f","Type":"ContainerStarted","Data":"4134ae236568a4bc741307b5fee9b15442f3ec804ae46f1874229ba9aba192e8"}
Sep 06 05:00:06.216548 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:06.216513    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vkvc2" event={"ID":"4d78dc73-4a38-43c7-865a-c74aeca7478f","Type":"ContainerStarted","Data":"fb6345ec689db12a5a6741a8a8442f15f7d5d0de62e6b34edee91427d04ce28e"}
Sep 06 05:00:06.232559 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:06.232515    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vkvc2" podStartSLOduration=17.86837708 podStartE2EDuration="21.232500349s" totalImagesPullingTime="3.364123269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:59:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:00:06.230985921 +0000 UTC m=+273.600441167" watchObservedRunningTime="2026-09-06 05:00:06.232500349 +0000 UTC m=+273.601955594"
Sep 06 05:00:07.168273 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:07.168243    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-l7kbk"]
Sep 06 05:00:07.170996 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:07.170980    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-l7kbk"
Sep 06 05:00:07.174107 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:07.174085    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 06 05:00:07.174107 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:07.174092    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bvb68\""
Sep 06 05:00:07.178590 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:07.178570    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-l7kbk"]
Sep 06 05:00:07.257346 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:07.257314    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:07.264006 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:07.263985    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e7a05bfc-b1b2-49d3-b48e-6e49a5a69581-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-l7kbk\" (UID: \"e7a05bfc-b1b2-49d3-b48e-6e49a5a69581\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-l7kbk"
Sep 06 05:00:07.354247 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:07.354218    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:07.365313 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:07.365275    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e7a05bfc-b1b2-49d3-b48e-6e49a5a69581-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-l7kbk\" (UID: \"e7a05bfc-b1b2-49d3-b48e-6e49a5a69581\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-l7kbk"
Sep 06 05:00:07.367592 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:07.367569    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e7a05bfc-b1b2-49d3-b48e-6e49a5a69581-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-l7kbk\" (UID: \"e7a05bfc-b1b2-49d3-b48e-6e49a5a69581\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-l7kbk"
Sep 06 05:00:07.479982 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:07.479904    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-l7kbk"
Sep 06 05:00:07.718169 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:07.718130    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:07.743820 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:07.743636    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-l7kbk"]
Sep 06 05:00:07.745997 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:00:07.745971    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a05bfc_b1b2_49d3_b48e_6e49a5a69581.slice/crio-5b6e5570497d262d1f15c413322c7896b6f9c711140a7f0d9164eb26af9d25c9 WatchSource:0}: Error finding container 5b6e5570497d262d1f15c413322c7896b6f9c711140a7f0d9164eb26af9d25c9: Status 404 returned error can't find the container with id 5b6e5570497d262d1f15c413322c7896b6f9c711140a7f0d9164eb26af9d25c9
Sep 06 05:00:07.803335 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:07.803309    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:08.224340 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:08.224297    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-l7kbk" event={"ID":"e7a05bfc-b1b2-49d3-b48e-6e49a5a69581","Type":"ContainerStarted","Data":"5b6e5570497d262d1f15c413322c7896b6f9c711140a7f0d9164eb26af9d25c9"}
Sep 06 05:00:09.228540 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:09.228506    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-l7kbk" event={"ID":"e7a05bfc-b1b2-49d3-b48e-6e49a5a69581","Type":"ContainerStarted","Data":"e0e3a5e15e9d7de08580f8b70fb26a10b2d6aaca6963f5eccac6952fd317a091"}
Sep 06 05:00:09.228909 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:09.228697    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-l7kbk"
Sep 06 05:00:09.233273 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:09.233252    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-l7kbk"
Sep 06 05:00:09.242110 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:09.242072    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-l7kbk" podStartSLOduration=1.311210888 podStartE2EDuration="2.242060057s" totalImagesPullingTime="930.849169ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:00:09.241456507 +0000 UTC m=+276.610911758" watchObservedRunningTime="2026-09-06 05:00:09.242060057 +0000 UTC m=+276.611515303"
Sep 06 05:00:09.660073 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:09.660042    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:09.749570 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:09.749541    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:09.895372 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:09.895338    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:10.325725 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:10.325690    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:10.405994 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:10.405959    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:12.121011 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:12.120979    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:13.795222 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:13.795192    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:13.893318 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:00:13.893277    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jzb7s" podUID="379e032e-b73c-47bf-ad03-faa4ffc31425"
Sep 06 05:00:13.914666 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:13.914637    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:14.021490 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.021458    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:14.052975 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.052902    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vp8qm"]
Sep 06 05:00:14.056035 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.056006    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.058526 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.058500    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 06 05:00:14.058721 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.058702    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6z6cl\""
Sep 06 05:00:14.059008 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.058993    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 06 05:00:14.059104 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.059027    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 06 05:00:14.059195 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.059182    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 06 05:00:14.059257 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.059243    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 06 05:00:14.059426 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.059410    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 06 05:00:14.101551 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.101525    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:14.163062 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.163035    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:14.220171 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.220140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-textfile\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.220325 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.220180    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-accelerators-collector-config\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.220325 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.220247    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/422d64d1-f6c6-4b05-8149-304fe879eb59-root\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.220325 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.220267    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-wtmp\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.220325 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.220292    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-tls\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.220325 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.220307    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.220481 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.220329    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422d64d1-f6c6-4b05-8149-304fe879eb59-sys\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.220481 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.220370    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndb2l\" (UniqueName: \"kubernetes.io/projected/422d64d1-f6c6-4b05-8149-304fe879eb59-kube-api-access-ndb2l\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.220481 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.220413    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/422d64d1-f6c6-4b05-8149-304fe879eb59-metrics-client-ca\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.243265 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.243240    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jzb7s"
Sep 06 05:00:14.266540 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.266505    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:14.321677 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.321585    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-tls\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.321677 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.321627    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.321677 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.321650    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422d64d1-f6c6-4b05-8149-304fe879eb59-sys\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.321939 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.321686    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndb2l\" (UniqueName: \"kubernetes.io/projected/422d64d1-f6c6-4b05-8149-304fe879eb59-kube-api-access-ndb2l\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.321939 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.321710    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/422d64d1-f6c6-4b05-8149-304fe879eb59-metrics-client-ca\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.321939 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.321726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-textfile\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.321939 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.321751    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-accelerators-collector-config\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.321939 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.321805    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/422d64d1-f6c6-4b05-8149-304fe879eb59-root\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.321939 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.321818    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422d64d1-f6c6-4b05-8149-304fe879eb59-sys\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.321939 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.321827    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-wtmp\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.321939 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.321881    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/422d64d1-f6c6-4b05-8149-304fe879eb59-root\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.322313 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.321987    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-wtmp\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.322438 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.322411    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/422d64d1-f6c6-4b05-8149-304fe879eb59-metrics-client-ca\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.322523 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.322411    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-accelerators-collector-config\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.322606 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.322588    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-textfile\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.323967 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.323950    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.324065 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.324035    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/422d64d1-f6c6-4b05-8149-304fe879eb59-node-exporter-tls\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.329285 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.329266    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndb2l\" (UniqueName: \"kubernetes.io/projected/422d64d1-f6c6-4b05-8149-304fe879eb59-kube-api-access-ndb2l\") pod \"node-exporter-vp8qm\" (UID: \"422d64d1-f6c6-4b05-8149-304fe879eb59\") " pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.365000 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.364965    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vp8qm"
Sep 06 05:00:14.365975 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.365954    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:14.377605 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:00:14.377572    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422d64d1_f6c6_4b05_8149_304fe879eb59.slice/crio-0b7b28fb3f956ddea563e604b1ddee5b6f909df5bcefbabbe820ddb56dbef7e4 WatchSource:0}: Error finding container 0b7b28fb3f956ddea563e604b1ddee5b6f909df5bcefbabbe820ddb56dbef7e4: Status 404 returned error can't find the container with id 0b7b28fb3f956ddea563e604b1ddee5b6f909df5bcefbabbe820ddb56dbef7e4
Sep 06 05:00:14.466465 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.466434    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:14.566776 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.566747    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:14.616386 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.616317    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:14.717096 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:14.717060    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:15.021519 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.021471    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:15.120788 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.120756    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:15.125172 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.125139    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 06 05:00:15.128916 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.128894    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.132271 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.132248    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 06 05:00:15.132404 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.132252    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 06 05:00:15.132566 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.132549    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9d8gk\""
Sep 06 05:00:15.132621 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.132609    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 06 05:00:15.132669 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.132647    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 06 05:00:15.132707 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.132252    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 06 05:00:15.132707 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.132549    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 06 05:00:15.132797 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.132775    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 06 05:00:15.132848 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.132798    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 06 05:00:15.132899 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.132865    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 06 05:00:15.144317 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.143527    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 06 05:00:15.228189 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.228148    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4bed4b02-e289-4389-acf9-72421ae759c5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.228357 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.228203    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.228357 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.228284    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-config-volume\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.228357 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.228316    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.228508 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.228374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.228508 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.228435    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.228508 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.228482    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.228655 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.228548    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.228655 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.228594    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-web-config\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.228655 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.228623    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.228770 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.228671    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqh56\" (UniqueName: \"kubernetes.io/projected/4bed4b02-e289-4389-acf9-72421ae759c5-kube-api-access-wqh56\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.228770 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.228704    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4bed4b02-e289-4389-acf9-72421ae759c5-config-out\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.228770 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.228724    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.237480 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.237445    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:15.247669 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.247639    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vp8qm" event={"ID":"422d64d1-f6c6-4b05-8149-304fe879eb59","Type":"ContainerStarted","Data":"0b7b28fb3f956ddea563e604b1ddee5b6f909df5bcefbabbe820ddb56dbef7e4"}
Sep 06 05:00:15.329340 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.329259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.329340 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.329334    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.329544 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.329364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.329544 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.329388    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.329544 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.329419    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-web-config\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.329544 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.329443    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.329544 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.329470    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqh56\" (UniqueName: \"kubernetes.io/projected/4bed4b02-e289-4389-acf9-72421ae759c5-kube-api-access-wqh56\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.329544 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.329495    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4bed4b02-e289-4389-acf9-72421ae759c5-config-out\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.329544 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.329521    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.329887 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.329550    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4bed4b02-e289-4389-acf9-72421ae759c5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.329887 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.329583    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.329887 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.329611    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-config-volume\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.329887 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.329635    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.330111 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.330060    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.330170 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:00:15.330143    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-trusted-ca-bundle podName:4bed4b02-e289-4389-acf9-72421ae759c5 nodeName:}" failed. No retries permitted until 2026-09-06 05:00:15.830116718 +0000 UTC m=+283.199571953 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "4bed4b02-e289-4389-acf9-72421ae759c5") : configmap references non-existent config key: ca-bundle.crt
Sep 06 05:00:15.330476 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.330401    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.332525 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.332500    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.333762 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.333717    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.334345 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.334322    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4bed4b02-e289-4389-acf9-72421ae759c5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.334447 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.334430    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-config-volume\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.335033 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.334838    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.335033 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.334855    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4bed4b02-e289-4389-acf9-72421ae759c5-config-out\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.335033 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.334988    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.335216 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.335091    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.335216 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.335115    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-web-config\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.338374 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.338353    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqh56\" (UniqueName: \"kubernetes.io/projected/4bed4b02-e289-4389-acf9-72421ae759c5-kube-api-access-wqh56\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.833589 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.833534    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:15.834338 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:15.834318    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:16.020858 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.020825    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"]
Sep 06 05:00:16.024314 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.024297    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.026779 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.026751    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 06 05:00:16.026779 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.026769    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-cdukank1dk5vu\""
Sep 06 05:00:16.026960 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.026860    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-b8gvl\""
Sep 06 05:00:16.026960 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.026874    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 06 05:00:16.027082 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.026970    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 06 05:00:16.027154 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.027107    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 06 05:00:16.027154 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.027115    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 06 05:00:16.034794 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.034770    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"]
Sep 06 05:00:16.040636 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.040617    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:00:16.136751 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.136722    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.136901 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.136793    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7xqv\" (UniqueName: \"kubernetes.io/projected/49d317ee-f7b8-4602-a9c5-486ad7a27a64-kube-api-access-m7xqv\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.136901 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.136839    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.136973 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.136903    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-tls\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.136973 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.136928    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49d317ee-f7b8-4602-a9c5-486ad7a27a64-metrics-client-ca\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.136973 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.136947    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-grpc-tls\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.137095 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.136976    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.137095 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.137027    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.237756 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.237721    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7xqv\" (UniqueName: \"kubernetes.io/projected/49d317ee-f7b8-4602-a9c5-486ad7a27a64-kube-api-access-m7xqv\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.237943 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.237761    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.237943 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.237789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-tls\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.237943 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.237805    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49d317ee-f7b8-4602-a9c5-486ad7a27a64-metrics-client-ca\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.237943 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.237821    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-grpc-tls\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.237943 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.237859    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.237943 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.237905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.238289 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.237947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.238796 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.238740    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49d317ee-f7b8-4602-a9c5-486ad7a27a64-metrics-client-ca\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.240826 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.240784    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-tls\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.240826 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.240816    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-grpc-tls\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.240967 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.240864    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.240967 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.240935    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.241103 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.241082    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.241143 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.241082    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49d317ee-f7b8-4602-a9c5-486ad7a27a64-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.245795 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.245769    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7xqv\" (UniqueName: \"kubernetes.io/projected/49d317ee-f7b8-4602-a9c5-486ad7a27a64-kube-api-access-m7xqv\") pod \"thanos-querier-6696d6fcb6-9j4th\" (UID: \"49d317ee-f7b8-4602-a9c5-486ad7a27a64\") " pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.252539 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.252518    2578 generic.go:356] "Generic (PLEG): container finished" podID="422d64d1-f6c6-4b05-8149-304fe879eb59" containerID="f91b8e15daf04dbd215c34f51ef64105f7a761147ce82484efbb0c195ff8e9e0" exitCode=0
Sep 06 05:00:16.252640 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.252587    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vp8qm" event={"ID":"422d64d1-f6c6-4b05-8149-304fe879eb59","Type":"ContainerDied","Data":"f91b8e15daf04dbd215c34f51ef64105f7a761147ce82484efbb0c195ff8e9e0"}
Sep 06 05:00:16.315324 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.315285    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 06 05:00:16.317154 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:00:16.317126    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bed4b02_e289_4389_acf9_72421ae759c5.slice/crio-de1a537c4d588a12e7cbd2a4d0dcb8a1c73deec9ac6e7c4ee80b890bcfe7a3c3 WatchSource:0}: Error finding container de1a537c4d588a12e7cbd2a4d0dcb8a1c73deec9ac6e7c4ee80b890bcfe7a3c3: Status 404 returned error can't find the container with id de1a537c4d588a12e7cbd2a4d0dcb8a1c73deec9ac6e7c4ee80b890bcfe7a3c3
Sep 06 05:00:16.333695 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.333673    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:16.605048 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.604692    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"]
Sep 06 05:00:16.607244 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:00:16.607220    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d317ee_f7b8_4602_a9c5_486ad7a27a64.slice/crio-a28e92135991cc56f929beb7300b9312062f69e0d663a14eccc2bc2a47ec4e79 WatchSource:0}: Error finding container a28e92135991cc56f929beb7300b9312062f69e0d663a14eccc2bc2a47ec4e79: Status 404 returned error can't find the container with id a28e92135991cc56f929beb7300b9312062f69e0d663a14eccc2bc2a47ec4e79
Sep 06 05:00:16.938879 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:16.938844    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:17.040614 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:17.040575    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:17.141180 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:17.141140    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:17.240965 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:17.240884    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:17.259453 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:17.259411    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vp8qm" event={"ID":"422d64d1-f6c6-4b05-8149-304fe879eb59","Type":"ContainerStarted","Data":"e4b3f3bca2f4bb58ad9fc3bacb3b6c0d3065ed7e0bb9eaf6e4eff4db2123af13"}
Sep 06 05:00:17.259602 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:17.259492    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vp8qm" event={"ID":"422d64d1-f6c6-4b05-8149-304fe879eb59","Type":"ContainerStarted","Data":"b7710d1c04db0fc6efcafea8dc78ac66ec2f9a17f7e74549ee99075c9397b0eb"}
Sep 06 05:00:17.261057 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:17.261010    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerStarted","Data":"de1a537c4d588a12e7cbd2a4d0dcb8a1c73deec9ac6e7c4ee80b890bcfe7a3c3"}
Sep 06 05:00:17.262306 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:17.262277    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th" event={"ID":"49d317ee-f7b8-4602-a9c5-486ad7a27a64","Type":"ContainerStarted","Data":"a28e92135991cc56f929beb7300b9312062f69e0d663a14eccc2bc2a47ec4e79"}
Sep 06 05:00:17.277728 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:17.277669    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vp8qm" podStartSLOduration=2.187068599 podStartE2EDuration="3.277649707s" totalImagesPullingTime="1.030245803s" totalInitContainerRuntime="60.335305ms" isStatefulPod=true podCreationTimestamp="2026-09-06 05:00:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:00:17.276833381 +0000 UTC m=+284.646288628" watchObservedRunningTime="2026-09-06 05:00:17.277649707 +0000 UTC m=+284.647104954"
Sep 06 05:00:17.447498 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:17.447463    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 05:00:17.449917 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:17.449894    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/379e032e-b73c-47bf-ad03-faa4ffc31425-metrics-tls\") pod \"dns-default-jzb7s\" (UID: \"379e032e-b73c-47bf-ad03-faa4ffc31425\") " pod="openshift-dns/dns-default-jzb7s"
Sep 06 05:00:17.546874 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:17.546793    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-prh2m\""
Sep 06 05:00:17.555568 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:17.555534    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jzb7s"
Sep 06 05:00:17.833122 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:17.833047    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jzb7s"]
Sep 06 05:00:17.835947 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:00:17.835914    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379e032e_b73c_47bf_ad03_faa4ffc31425.slice/crio-af40ffca11e605b64554bc6954b66e0f799a6a952982df25c48ac3041364f715 WatchSource:0}: Error finding container af40ffca11e605b64554bc6954b66e0f799a6a952982df25c48ac3041364f715: Status 404 returned error can't find the container with id af40ffca11e605b64554bc6954b66e0f799a6a952982df25c48ac3041364f715
Sep 06 05:00:18.267547 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:18.267510    2578 generic.go:356] "Generic (PLEG): container finished" podID="4bed4b02-e289-4389-acf9-72421ae759c5" containerID="715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62" exitCode=0
Sep 06 05:00:18.267985 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:18.267597    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerDied","Data":"715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62"}
Sep 06 05:00:18.269052 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:18.269007    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jzb7s" event={"ID":"379e032e-b73c-47bf-ad03-faa4ffc31425","Type":"ContainerStarted","Data":"af40ffca11e605b64554bc6954b66e0f799a6a952982df25c48ac3041364f715"}
Sep 06 05:00:18.813657 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:18.813601    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:19.080143 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:19.080056    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:19.276467 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:19.276432    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th" event={"ID":"49d317ee-f7b8-4602-a9c5-486ad7a27a64","Type":"ContainerStarted","Data":"e94ba9a0ea45f6591bc84e8d6682071881c42bab78c8073008d6bb5011605dd6"}
Sep 06 05:00:19.276860 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:19.276511    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th" event={"ID":"49d317ee-f7b8-4602-a9c5-486ad7a27a64","Type":"ContainerStarted","Data":"780c82442124d9edb86c6e4d4158d637b426527bceefbc99e014bf174847ad96"}
Sep 06 05:00:19.276860 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:19.276558    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th" event={"ID":"49d317ee-f7b8-4602-a9c5-486ad7a27a64","Type":"ContainerStarted","Data":"b8e2ca67196e1470a291c14be2792158eea877f49761edb816ca68462a0b46cf"}
Sep 06 05:00:20.252362 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:20.252336    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:21.289729 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.289687    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerStarted","Data":"4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd"}
Sep 06 05:00:21.290186 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.289770    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerStarted","Data":"5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0"}
Sep 06 05:00:21.290186 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.289810    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerStarted","Data":"e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e"}
Sep 06 05:00:21.290186 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.289846    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerStarted","Data":"4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa"}
Sep 06 05:00:21.290186 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.289882    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerStarted","Data":"612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36"}
Sep 06 05:00:21.290186 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.289917    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerStarted","Data":"9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec"}
Sep 06 05:00:21.292641 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.292613    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th" event={"ID":"49d317ee-f7b8-4602-a9c5-486ad7a27a64","Type":"ContainerStarted","Data":"347cc655f9be440399a359f4eb70dc045c0e07d8f3d2ded2b4cec087d708a5e4"}
Sep 06 05:00:21.292741 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.292683    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th" event={"ID":"49d317ee-f7b8-4602-a9c5-486ad7a27a64","Type":"ContainerStarted","Data":"cb02d42baab38be19fe551b1bc46eae76016243da3015b163c94af8d1ddd96b9"}
Sep 06 05:00:21.292741 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.292727    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th" event={"ID":"49d317ee-f7b8-4602-a9c5-486ad7a27a64","Type":"ContainerStarted","Data":"fe94a8d18ebec743fbcef85343e3b8b635a70bdeaf215966be8bdf3af7698027"}
Sep 06 05:00:21.292838 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.292770    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:21.294460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.294438    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jzb7s" event={"ID":"379e032e-b73c-47bf-ad03-faa4ffc31425","Type":"ContainerStarted","Data":"6f4f4bf5eb992d8ce6c6dbc5d3b6a45da86587217d8dba8df4e6cc97ddb551c7"}
Sep 06 05:00:21.294577 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.294475    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jzb7s" event={"ID":"379e032e-b73c-47bf-ad03-faa4ffc31425","Type":"ContainerStarted","Data":"d091b9f9e3df0fcbe5127e74ef74c0ffe2b91f5b74953f05379a515ff0e918a8"}
Sep 06 05:00:21.294577 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.294517    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jzb7s"
Sep 06 05:00:21.314724 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.314676    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.307386921 podStartE2EDuration="6.314662019s" totalImagesPullingTime="2.988977891s" totalInitContainerRuntime="18.297207ms" isStatefulPod=false podCreationTimestamp="2026-09-06 05:00:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:00:21.312851354 +0000 UTC m=+288.682306600" watchObservedRunningTime="2026-09-06 05:00:21.314662019 +0000 UTC m=+288.684117264"
Sep 06 05:00:21.335887 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.335846    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th" podStartSLOduration=1.845896854 podStartE2EDuration="5.335832042s" totalImagesPullingTime="3.489935188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:00:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:00:21.334400801 +0000 UTC m=+288.703856048" watchObservedRunningTime="2026-09-06 05:00:21.335832042 +0000 UTC m=+288.705287287"
Sep 06 05:00:21.349298 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:21.349254    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jzb7s" podStartSLOduration=251.957414954 podStartE2EDuration="4m14.349237515s" totalImagesPullingTime="2.391822561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:56:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:00:21.348460123 +0000 UTC m=+288.717915368" watchObservedRunningTime="2026-09-06 05:00:21.349237515 +0000 UTC m=+288.718692764"
Sep 06 05:00:22.635866 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:22.635835    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:22.738213 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:22.738176    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:27.304719 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:27.304691    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6696d6fcb6-9j4th"
Sep 06 05:00:29.097081 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:29.097049    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:29.197343 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:29.197309    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:29.444267 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:29.444234    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:29.544916 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:29.544881    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:29.594212 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:29.594175    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:29.742069 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:29.741966    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:29.842813 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:29.842781    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:29.943364 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:29.943334    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:30.149037 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:00:30.148975    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-dxn9s" podUID="d85cbcaa-00ae-4440-b7f0-7209f409a5cd"
Sep 06 05:00:31.300748 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:31.300720    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jzb7s"
Sep 06 05:00:31.344300 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:31.344252    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:31.396381 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:31.396353    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:33.060839 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:33.060812    2578 kubelet.go:1648] "Image garbage collection succeeded"
Sep 06 05:00:39.910246 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:39.910201    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-8jjn2"]
Sep 06 05:00:39.916854 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:39.916835    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-8jjn2"
Sep 06 05:00:39.919104 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:39.919087    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 06 05:00:39.919104 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:39.919092    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 06 05:00:39.919243 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:39.919092    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ctl6j\""
Sep 06 05:00:39.923738 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:39.923586    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-8jjn2"]
Sep 06 05:00:39.924906 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:39.924882    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:40.023480 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:40.023447    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z24hg\" (UniqueName: \"kubernetes.io/projected/0ce339fc-a3c7-40bf-8d50-384552a8da06-kube-api-access-z24hg\") pod \"downloads-85d98c998f-8jjn2\" (UID: \"0ce339fc-a3c7-40bf-8d50-384552a8da06\") " pod="openshift-console/downloads-85d98c998f-8jjn2"
Sep 06 05:00:40.023639 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:40.023602    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:40.124446 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:40.124417    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z24hg\" (UniqueName: \"kubernetes.io/projected/0ce339fc-a3c7-40bf-8d50-384552a8da06-kube-api-access-z24hg\") pod \"downloads-85d98c998f-8jjn2\" (UID: \"0ce339fc-a3c7-40bf-8d50-384552a8da06\") " pod="openshift-console/downloads-85d98c998f-8jjn2"
Sep 06 05:00:40.131999 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:40.131975    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z24hg\" (UniqueName: \"kubernetes.io/projected/0ce339fc-a3c7-40bf-8d50-384552a8da06-kube-api-access-z24hg\") pod \"downloads-85d98c998f-8jjn2\" (UID: \"0ce339fc-a3c7-40bf-8d50-384552a8da06\") " pod="openshift-console/downloads-85d98c998f-8jjn2"
Sep 06 05:00:40.210883 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:40.210812    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:40.226946 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:40.226909    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-8jjn2"
Sep 06 05:00:40.416155 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:40.416124    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:40.467857 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:40.467771    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:40.496866 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:40.496839    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-8jjn2"]
Sep 06 05:00:40.499811 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:00:40.499783    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ce339fc_a3c7_40bf_8d50_384552a8da06.slice/crio-b80978661e95dd9bbd1065f63aa5213dec7314cbee5dfde80bb6c1a93b416c43 WatchSource:0}: Error finding container b80978661e95dd9bbd1065f63aa5213dec7314cbee5dfde80bb6c1a93b416c43: Status 404 returned error can't find the container with id b80978661e95dd9bbd1065f63aa5213dec7314cbee5dfde80bb6c1a93b416c43
Sep 06 05:00:40.502245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:40.502227    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 05:00:40.599407 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:40.599375    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:40.649593 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:40.649565    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:41.003850 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:41.003814    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:41.057817 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:41.057782    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:41.147878 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:41.147844    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 05:00:41.221694 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:41.221642    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:41.368225 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:41.368142    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-8jjn2" event={"ID":"0ce339fc-a3c7-40bf-8d50-384552a8da06","Type":"ContainerStarted","Data":"b80978661e95dd9bbd1065f63aa5213dec7314cbee5dfde80bb6c1a93b416c43"}
Sep 06 05:00:41.400608 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:41.400563    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:41.802385 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:41.802260    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:42.311925 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:42.311890    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:42.811754 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:42.811709    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:42.858773 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:42.858748    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:43.024945 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:43.024894    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:43.126324 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:43.126292    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:45.597804 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.597768    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-676dcc8677-fkcsj"]
Sep 06 05:00:45.601886 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.601862    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.604395 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.604215    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 06 05:00:45.604395 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.604268    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 06 05:00:45.604395 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.604283    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 06 05:00:45.605534 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.605446    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 06 05:00:45.605534 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.605461    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-b6968\""
Sep 06 05:00:45.605534 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.605477    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 06 05:00:45.609960 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.609903    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-676dcc8677-fkcsj"]
Sep 06 05:00:45.668076 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.668045    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97885\" (UniqueName: \"kubernetes.io/projected/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-kube-api-access-97885\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.668245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.668086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-serving-cert\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.668245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.668142    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-oauth-config\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.668245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.668159    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-service-ca\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.668245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.668177    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-oauth-serving-cert\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.668420 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.668321    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-config\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.769489 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.769450    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-config\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.769489 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.769500    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97885\" (UniqueName: \"kubernetes.io/projected/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-kube-api-access-97885\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.769756 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.769521    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-serving-cert\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.769756 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.769550    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-oauth-config\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.769756 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.769576    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-service-ca\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.769756 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.769599    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-oauth-serving-cert\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.770445 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.770421    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-oauth-serving-cert\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.770445 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.770440    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-config\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.770626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.770465    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-service-ca\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.772440 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.772418    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-oauth-config\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.772564 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.772483    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-serving-cert\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.777236 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.777212    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97885\" (UniqueName: \"kubernetes.io/projected/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-kube-api-access-97885\") pod \"console-676dcc8677-fkcsj\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") " pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:45.914826 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:45.914792    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:46.196503 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:46.196476    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-676dcc8677-fkcsj"]
Sep 06 05:00:46.199199 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:00:46.199170    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78f545e2_7d86_44ff_9934_ed5c0bfe8c58.slice/crio-0b6a683c17cbdf3b953642a06da2a9c4e9947fb7e0e0253b2232833d266a3cf3 WatchSource:0}: Error finding container 0b6a683c17cbdf3b953642a06da2a9c4e9947fb7e0e0253b2232833d266a3cf3: Status 404 returned error can't find the container with id 0b6a683c17cbdf3b953642a06da2a9c4e9947fb7e0e0253b2232833d266a3cf3
Sep 06 05:00:46.388685 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:46.388641    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676dcc8677-fkcsj" event={"ID":"78f545e2-7d86-44ff-9934-ed5c0bfe8c58","Type":"ContainerStarted","Data":"0b6a683c17cbdf3b953642a06da2a9c4e9947fb7e0e0253b2232833d266a3cf3"}
Sep 06 05:00:49.405882 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:49.405845    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676dcc8677-fkcsj" event={"ID":"78f545e2-7d86-44ff-9934-ed5c0bfe8c58","Type":"ContainerStarted","Data":"2a6143b1b35617d7b2b937ea49bdf089776a41fa577d615bdfdfbe72ef7aecec"}
Sep 06 05:00:49.422836 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:49.422778    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-676dcc8677-fkcsj" podStartSLOduration=1.323431186 podStartE2EDuration="4.422763147s" totalImagesPullingTime="3.099331961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:00:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:00:49.421450329 +0000 UTC m=+316.790905575" watchObservedRunningTime="2026-09-06 05:00:49.422763147 +0000 UTC m=+316.792218392"
Sep 06 05:00:49.429343 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:49.429314    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:49.525068 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:49.524977    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:50.122676 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:50.122621    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh" podUID="bf76b39e-fe5f-4d1d-9e0e-6177bf241302" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 06 05:00:51.259113 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:51.259075    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:51.315805 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:51.315760    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:51.874074 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:51.874041    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:51.926277 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:51.926238    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:52.642458 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:52.642410    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:52.740407 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:52.740363    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:54.030923 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:54.030866    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:54.134301 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:54.134260    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:55.915458 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:55.915422    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:55.915913 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:55.915500    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:55.920773 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:55.920747    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:56.436203 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:56.436165    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-8jjn2" event={"ID":"0ce339fc-a3c7-40bf-8d50-384552a8da06","Type":"ContainerStarted","Data":"359eaadb3f5b869299db9be350c59976f492c3c76b8a9715b37eb02d08175844"}
Sep 06 05:00:56.440199 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:56.440172    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:00:56.451251 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:56.451210    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-8jjn2" podStartSLOduration=1.645511633 podStartE2EDuration="17.451194478s" totalImagesPullingTime="15.805682845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:00:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:00:56.449749167 +0000 UTC m=+323.819204439" watchObservedRunningTime="2026-09-06 05:00:56.451194478 +0000 UTC m=+323.820649724"
Sep 06 05:00:57.429153 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:57.429110    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:00:57.440798 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:57.440747    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-8jjn2"
Sep 06 05:00:57.459872 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:57.459845    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-8jjn2"
Sep 06 05:00:57.486171 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:00:57.486139    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:00.122605 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:00.122563    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh" podUID="bf76b39e-fe5f-4d1d-9e0e-6177bf241302" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 06 05:01:01.506921 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:01.506890    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4bed4b02-e289-4389-acf9-72421ae759c5/init-config-reloader/0.log"
Sep 06 05:01:01.516808 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:01.516781    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4bed4b02-e289-4389-acf9-72421ae759c5/alertmanager/0.log"
Sep 06 05:01:01.588584 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:01.588558    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4bed4b02-e289-4389-acf9-72421ae759c5/config-reloader/0.log"
Sep 06 05:01:01.787521 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:01.787440    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4bed4b02-e289-4389-acf9-72421ae759c5/kube-rbac-proxy-web/0.log"
Sep 06 05:01:01.987108 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:01.987076    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4bed4b02-e289-4389-acf9-72421ae759c5/kube-rbac-proxy/0.log"
Sep 06 05:01:02.187727 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:02.187693    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4bed4b02-e289-4389-acf9-72421ae759c5/kube-rbac-proxy-metric/0.log"
Sep 06 05:01:02.387533 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:02.387499    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4bed4b02-e289-4389-acf9-72421ae759c5/prom-label-proxy/0.log"
Sep 06 05:01:02.836784 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:02.836742    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:04.987061 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:04.987027    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vp8qm_422d64d1-f6c6-4b05-8149-304fe879eb59/init-textfile/0.log"
Sep 06 05:01:05.187608 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:05.187574    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vp8qm_422d64d1-f6c6-4b05-8149-304fe879eb59/node-exporter/0.log"
Sep 06 05:01:05.387566 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:05.387534    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vp8qm_422d64d1-f6c6-4b05-8149-304fe879eb59/kube-rbac-proxy/0.log"
Sep 06 05:01:05.436262 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:05.436229    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:05.515440 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:05.515407    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-676dcc8677-fkcsj"]
Sep 06 05:01:06.430394 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:06.430358    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:06.481722 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:06.481690    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:07.432676 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:07.432644    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:07.533189 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:07.533161    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:07.637437 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:07.637408    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:07.685339 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:07.685273    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:07.986565 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:07.986500    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-l7kbk_e7a05bfc-b1b2-49d3-b48e-6e49a5a69581/prometheus-operator-admission-webhook/0.log"
Sep 06 05:01:08.787462 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:08.787432    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6696d6fcb6-9j4th_49d317ee-f7b8-4602-a9c5-486ad7a27a64/thanos-query/0.log"
Sep 06 05:01:08.987004 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:08.986977    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6696d6fcb6-9j4th_49d317ee-f7b8-4602-a9c5-486ad7a27a64/kube-rbac-proxy-web/0.log"
Sep 06 05:01:09.186751 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:09.186730    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6696d6fcb6-9j4th_49d317ee-f7b8-4602-a9c5-486ad7a27a64/kube-rbac-proxy/0.log"
Sep 06 05:01:09.387287 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:09.387261    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6696d6fcb6-9j4th_49d317ee-f7b8-4602-a9c5-486ad7a27a64/prom-label-proxy/0.log"
Sep 06 05:01:09.587493 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:09.587420    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6696d6fcb6-9j4th_49d317ee-f7b8-4602-a9c5-486ad7a27a64/kube-rbac-proxy-rules/0.log"
Sep 06 05:01:09.788011 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:09.787960    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6696d6fcb6-9j4th_49d317ee-f7b8-4602-a9c5-486ad7a27a64/kube-rbac-proxy-metrics/0.log"
Sep 06 05:01:10.122056 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:10.122008    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh" podUID="bf76b39e-fe5f-4d1d-9e0e-6177bf241302" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 06 05:01:10.122218 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:10.122088    2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh"
Sep 06 05:01:10.122588 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:10.122559    2578 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"6e4885bf0c089a9451304430d83a7ab9d1a847a3709a22bf4cd41529bef4ad97"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 06 05:01:10.122630 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:10.122609    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh" podUID="bf76b39e-fe5f-4d1d-9e0e-6177bf241302" containerName="service-proxy" containerID="cri-o://6e4885bf0c089a9451304430d83a7ab9d1a847a3709a22bf4cd41529bef4ad97" gracePeriod=30
Sep 06 05:01:10.491502 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:10.491425    2578 generic.go:356] "Generic (PLEG): container finished" podID="bf76b39e-fe5f-4d1d-9e0e-6177bf241302" containerID="6e4885bf0c089a9451304430d83a7ab9d1a847a3709a22bf4cd41529bef4ad97" exitCode=2
Sep 06 05:01:10.491502 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:10.491466    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh" event={"ID":"bf76b39e-fe5f-4d1d-9e0e-6177bf241302","Type":"ContainerDied","Data":"6e4885bf0c089a9451304430d83a7ab9d1a847a3709a22bf4cd41529bef4ad97"}
Sep 06 05:01:10.491502 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:10.491496    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cbf55c7c8-n24fh" event={"ID":"bf76b39e-fe5f-4d1d-9e0e-6177bf241302","Type":"ContainerStarted","Data":"90f68b033d7fbaea3efc5bd57158ea4f51115255aa5c8c1adef18c0660ed5a9e"}
Sep 06 05:01:10.587291 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:10.587261    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-676dcc8677-fkcsj_78f545e2-7d86-44ff-9934-ed5c0bfe8c58/console/0.log"
Sep 06 05:01:10.988460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:10.988434    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-8jjn2_0ce339fc-a3c7-40bf-8d50-384552a8da06/download-server/0.log"
Sep 06 05:01:22.640888 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:22.640843    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:22.741302 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:22.741269    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:30.087904 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.087815    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:30.141684 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.141656    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:30.534972 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.534933    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-676dcc8677-fkcsj" podUID="78f545e2-7d86-44ff-9934-ed5c0bfe8c58" containerName="console" containerID="cri-o://2a6143b1b35617d7b2b937ea49bdf089776a41fa577d615bdfdfbe72ef7aecec" gracePeriod=15
Sep 06 05:01:30.805136 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.805072    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:01:30.853544 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.853514    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97885\" (UniqueName: \"kubernetes.io/projected/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-kube-api-access-97885\") pod \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") "
Sep 06 05:01:30.853544 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.853553    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-oauth-serving-cert\") pod \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") "
Sep 06 05:01:30.853795 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.853576    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-service-ca\") pod \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") "
Sep 06 05:01:30.853795 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.853599    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-oauth-config\") pod \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") "
Sep 06 05:01:30.853795 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.853682    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-config\") pod \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") "
Sep 06 05:01:30.853795 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.853792    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-serving-cert\") pod \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\" (UID: \"78f545e2-7d86-44ff-9934-ed5c0bfe8c58\") "
Sep 06 05:01:30.854056 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.854011    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "78f545e2-7d86-44ff-9934-ed5c0bfe8c58" (UID: "78f545e2-7d86-44ff-9934-ed5c0bfe8c58"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:01:30.854056 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.854043    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-service-ca" (OuterVolumeSpecName: "service-ca") pod "78f545e2-7d86-44ff-9934-ed5c0bfe8c58" (UID: "78f545e2-7d86-44ff-9934-ed5c0bfe8c58"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:01:30.854164 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.854092    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-config" (OuterVolumeSpecName: "console-config") pod "78f545e2-7d86-44ff-9934-ed5c0bfe8c58" (UID: "78f545e2-7d86-44ff-9934-ed5c0bfe8c58"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:01:30.855840 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.855815    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-kube-api-access-97885" (OuterVolumeSpecName: "kube-api-access-97885") pod "78f545e2-7d86-44ff-9934-ed5c0bfe8c58" (UID: "78f545e2-7d86-44ff-9934-ed5c0bfe8c58"). InnerVolumeSpecName "kube-api-access-97885". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:01:30.855931 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.855832    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "78f545e2-7d86-44ff-9934-ed5c0bfe8c58" (UID: "78f545e2-7d86-44ff-9934-ed5c0bfe8c58"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:30.855931 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.855894    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "78f545e2-7d86-44ff-9934-ed5c0bfe8c58" (UID: "78f545e2-7d86-44ff-9934-ed5c0bfe8c58"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:30.954735 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.954693    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-config\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:30.954735 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.954724    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-serving-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:30.954735 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.954734    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97885\" (UniqueName: \"kubernetes.io/projected/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-kube-api-access-97885\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:30.954735 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.954744    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-oauth-serving-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:30.954976 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.954753    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-service-ca\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:30.954976 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:30.954761    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78f545e2-7d86-44ff-9934-ed5c0bfe8c58-console-oauth-config\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:31.574082 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:31.573982    2578 generic.go:356] "Generic (PLEG): container finished" podID="78f545e2-7d86-44ff-9934-ed5c0bfe8c58" containerID="2a6143b1b35617d7b2b937ea49bdf089776a41fa577d615bdfdfbe72ef7aecec" exitCode=0
Sep 06 05:01:31.574082 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:31.574052    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676dcc8677-fkcsj"
Sep 06 05:01:31.574586 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:31.574072    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676dcc8677-fkcsj" event={"ID":"78f545e2-7d86-44ff-9934-ed5c0bfe8c58","Type":"ContainerDied","Data":"2a6143b1b35617d7b2b937ea49bdf089776a41fa577d615bdfdfbe72ef7aecec"}
Sep 06 05:01:31.574586 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:31.574138    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676dcc8677-fkcsj" event={"ID":"78f545e2-7d86-44ff-9934-ed5c0bfe8c58","Type":"ContainerDied","Data":"0b6a683c17cbdf3b953642a06da2a9c4e9947fb7e0e0253b2232833d266a3cf3"}
Sep 06 05:01:31.574586 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:31.574167    2578 scope.go:117] "RemoveContainer" containerID="2a6143b1b35617d7b2b937ea49bdf089776a41fa577d615bdfdfbe72ef7aecec"
Sep 06 05:01:31.582366 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:31.582346    2578 scope.go:117] "RemoveContainer" containerID="2a6143b1b35617d7b2b937ea49bdf089776a41fa577d615bdfdfbe72ef7aecec"
Sep 06 05:01:31.582648 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:01:31.582626    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a6143b1b35617d7b2b937ea49bdf089776a41fa577d615bdfdfbe72ef7aecec\": container with ID starting with 2a6143b1b35617d7b2b937ea49bdf089776a41fa577d615bdfdfbe72ef7aecec not found: ID does not exist" containerID="2a6143b1b35617d7b2b937ea49bdf089776a41fa577d615bdfdfbe72ef7aecec"
Sep 06 05:01:31.582700 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:31.582656    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a6143b1b35617d7b2b937ea49bdf089776a41fa577d615bdfdfbe72ef7aecec"} err="failed to get container status \"2a6143b1b35617d7b2b937ea49bdf089776a41fa577d615bdfdfbe72ef7aecec\": rpc error: code = NotFound desc = could not find container \"2a6143b1b35617d7b2b937ea49bdf089776a41fa577d615bdfdfbe72ef7aecec\": container with ID starting with 2a6143b1b35617d7b2b937ea49bdf089776a41fa577d615bdfdfbe72ef7aecec not found: ID does not exist"
Sep 06 05:01:31.589368 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:31.589347    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-676dcc8677-fkcsj"]
Sep 06 05:01:31.594147 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:31.594127    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-676dcc8677-fkcsj"]
Sep 06 05:01:32.171422 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.171372    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:32.231236 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.231203    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:32.753002 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.752963    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69fdc6d5bd-b4dmg"]
Sep 06 05:01:32.753479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.753375    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78f545e2-7d86-44ff-9934-ed5c0bfe8c58" containerName="console"
Sep 06 05:01:32.753479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.753393    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="78f545e2-7d86-44ff-9934-ed5c0bfe8c58" containerName="console"
Sep 06 05:01:32.753479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.753471    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="78f545e2-7d86-44ff-9934-ed5c0bfe8c58" containerName="console"
Sep 06 05:01:32.779769 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.779736    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69fdc6d5bd-b4dmg"]
Sep 06 05:01:32.779924 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.779855    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.782218 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.782195    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 06 05:01:32.783167 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.783126    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 06 05:01:32.783279 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.783199    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 06 05:01:32.783279 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.783199    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 06 05:01:32.783279 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.783199    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 06 05:01:32.783434 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.783200    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-b6968\""
Sep 06 05:01:32.787725 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.787706    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 06 05:01:32.804960 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.804938    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:32.858307 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.858278    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:32.868285 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.868261    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-service-ca\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.868385 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.868299    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-trusted-ca-bundle\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.868385 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.868331    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28x5z\" (UniqueName: \"kubernetes.io/projected/3be4d2db-df80-4f74-8297-3b2428250eaa-kube-api-access-28x5z\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.868493 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.868387    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-oauth-serving-cert\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.868493 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.868477    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-console-config\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.868591 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.868535    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be4d2db-df80-4f74-8297-3b2428250eaa-console-serving-cert\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.868591 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.868567    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be4d2db-df80-4f74-8297-3b2428250eaa-console-oauth-config\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.940853 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.940820    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:32.969194 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.969171    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-console-config\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.969266 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.969216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be4d2db-df80-4f74-8297-3b2428250eaa-console-serving-cert\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.969305 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.969266    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be4d2db-df80-4f74-8297-3b2428250eaa-console-oauth-config\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.969354 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.969319    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-service-ca\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.969354 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.969346    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-trusted-ca-bundle\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.969452 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.969374    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28x5z\" (UniqueName: \"kubernetes.io/projected/3be4d2db-df80-4f74-8297-3b2428250eaa-kube-api-access-28x5z\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.969452 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.969403    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-oauth-serving-cert\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.970049 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.970011    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-service-ca\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.970214 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.970194    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-console-config\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.970485 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.970470    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-oauth-serving-cert\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.970728 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.970706    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-trusted-ca-bundle\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.971756 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.971731    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be4d2db-df80-4f74-8297-3b2428250eaa-console-serving-cert\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.971949 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.971931    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be4d2db-df80-4f74-8297-3b2428250eaa-console-oauth-config\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:32.979643 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:32.979625    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28x5z\" (UniqueName: \"kubernetes.io/projected/3be4d2db-df80-4f74-8297-3b2428250eaa-kube-api-access-28x5z\") pod \"console-69fdc6d5bd-b4dmg\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") " pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:33.041201 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:33.041130    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:33.091457 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:33.091427    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-b6968\""
Sep 06 05:01:33.099557 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:33.099526    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:33.153208 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:33.153182    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78f545e2-7d86-44ff-9934-ed5c0bfe8c58" path="/var/lib/kubelet/pods/78f545e2-7d86-44ff-9934-ed5c0bfe8c58/volumes"
Sep 06 05:01:33.402391 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:33.402355    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69fdc6d5bd-b4dmg"]
Sep 06 05:01:33.405330 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:01:33.405305    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3be4d2db_df80_4f74_8297_3b2428250eaa.slice/crio-acda2c7e22541fb055bd1892a1ac1538ad00104393c32178954b6e27b43ede7f WatchSource:0}: Error finding container acda2c7e22541fb055bd1892a1ac1538ad00104393c32178954b6e27b43ede7f: Status 404 returned error can't find the container with id acda2c7e22541fb055bd1892a1ac1538ad00104393c32178954b6e27b43ede7f
Sep 06 05:01:33.551438 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:33.550916    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:33.584009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:33.583976    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69fdc6d5bd-b4dmg" event={"ID":"3be4d2db-df80-4f74-8297-3b2428250eaa","Type":"ContainerStarted","Data":"67c3d6e64c6e6f0e5508d088c10cc94c42523e9f099f3ef3499f34e33f1b1ee1"}
Sep 06 05:01:33.584189 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:33.584056    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69fdc6d5bd-b4dmg" event={"ID":"3be4d2db-df80-4f74-8297-3b2428250eaa","Type":"ContainerStarted","Data":"acda2c7e22541fb055bd1892a1ac1538ad00104393c32178954b6e27b43ede7f"}
Sep 06 05:01:33.600586 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:33.600531    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69fdc6d5bd-b4dmg" podStartSLOduration=1.600515432 podStartE2EDuration="1.600515432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:01:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:01:33.599331818 +0000 UTC m=+360.968787090" watchObservedRunningTime="2026-09-06 05:01:33.600515432 +0000 UTC m=+360.969970743"
Sep 06 05:01:33.601103 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:33.601079    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:34.295709 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.295673    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 06 05:01:34.296137 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.295942    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="alertmanager" containerID="cri-o://9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec" gracePeriod=120
Sep 06 05:01:34.296137 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.296058    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="prom-label-proxy" containerID="cri-o://4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd" gracePeriod=120
Sep 06 05:01:34.296137 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.296079    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="config-reloader" containerID="cri-o://612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36" gracePeriod=120
Sep 06 05:01:34.296137 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.296089    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="kube-rbac-proxy" containerID="cri-o://e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e" gracePeriod=120
Sep 06 05:01:34.296137 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.296066    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="kube-rbac-proxy-web" containerID="cri-o://4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa" gracePeriod=120
Sep 06 05:01:34.296354 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.296065    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="kube-rbac-proxy-metric" containerID="cri-o://5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0" gracePeriod=120
Sep 06 05:01:34.590966 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.590881    2578 generic.go:356] "Generic (PLEG): container finished" podID="4bed4b02-e289-4389-acf9-72421ae759c5" containerID="4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd" exitCode=0
Sep 06 05:01:34.590966 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.590906    2578 generic.go:356] "Generic (PLEG): container finished" podID="4bed4b02-e289-4389-acf9-72421ae759c5" containerID="e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e" exitCode=0
Sep 06 05:01:34.590966 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.590912    2578 generic.go:356] "Generic (PLEG): container finished" podID="4bed4b02-e289-4389-acf9-72421ae759c5" containerID="612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36" exitCode=0
Sep 06 05:01:34.590966 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.590919    2578 generic.go:356] "Generic (PLEG): container finished" podID="4bed4b02-e289-4389-acf9-72421ae759c5" containerID="9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec" exitCode=0
Sep 06 05:01:34.590966 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.590958    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerDied","Data":"4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd"}
Sep 06 05:01:34.591227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.591042    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerDied","Data":"e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e"}
Sep 06 05:01:34.591227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.591081    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerDied","Data":"612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36"}
Sep 06 05:01:34.591227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:34.591122    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerDied","Data":"9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec"}
Sep 06 05:01:35.553856 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.553826    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.588211 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.588179    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-metrics-client-ca\") pod \"4bed4b02-e289-4389-acf9-72421ae759c5\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") "
Sep 06 05:01:35.588381 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.588225    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4bed4b02-e289-4389-acf9-72421ae759c5-tls-assets\") pod \"4bed4b02-e289-4389-acf9-72421ae759c5\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") "
Sep 06 05:01:35.588381 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.588253    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4bed4b02-e289-4389-acf9-72421ae759c5\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") "
Sep 06 05:01:35.588381 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.588305    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-trusted-ca-bundle\") pod \"4bed4b02-e289-4389-acf9-72421ae759c5\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") "
Sep 06 05:01:35.588381 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.588335    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-main-db\") pod \"4bed4b02-e289-4389-acf9-72421ae759c5\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") "
Sep 06 05:01:35.588381 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.588365    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-cluster-tls-config\") pod \"4bed4b02-e289-4389-acf9-72421ae759c5\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") "
Sep 06 05:01:35.588644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.588396    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy\") pod \"4bed4b02-e289-4389-acf9-72421ae759c5\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") "
Sep 06 05:01:35.588644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.588432    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4bed4b02-e289-4389-acf9-72421ae759c5-config-out\") pod \"4bed4b02-e289-4389-acf9-72421ae759c5\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") "
Sep 06 05:01:35.588644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.588471    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy-web\") pod \"4bed4b02-e289-4389-acf9-72421ae759c5\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") "
Sep 06 05:01:35.588644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.588506    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqh56\" (UniqueName: \"kubernetes.io/projected/4bed4b02-e289-4389-acf9-72421ae759c5-kube-api-access-wqh56\") pod \"4bed4b02-e289-4389-acf9-72421ae759c5\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") "
Sep 06 05:01:35.588644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.588530    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-main-tls\") pod \"4bed4b02-e289-4389-acf9-72421ae759c5\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") "
Sep 06 05:01:35.588644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.588537    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4bed4b02-e289-4389-acf9-72421ae759c5" (UID: "4bed4b02-e289-4389-acf9-72421ae759c5"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:01:35.588644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.588566    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-config-volume\") pod \"4bed4b02-e289-4389-acf9-72421ae759c5\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") "
Sep 06 05:01:35.588644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.588597    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-web-config\") pod \"4bed4b02-e289-4389-acf9-72421ae759c5\" (UID: \"4bed4b02-e289-4389-acf9-72421ae759c5\") "
Sep 06 05:01:35.589871 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.589586    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4bed4b02-e289-4389-acf9-72421ae759c5" (UID: "4bed4b02-e289-4389-acf9-72421ae759c5"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:01:35.589871 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.589595    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-metrics-client-ca\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:35.590559 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.590532    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4bed4b02-e289-4389-acf9-72421ae759c5" (UID: "4bed4b02-e289-4389-acf9-72421ae759c5"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:01:35.593727 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.593683    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bed4b02-e289-4389-acf9-72421ae759c5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4bed4b02-e289-4389-acf9-72421ae759c5" (UID: "4bed4b02-e289-4389-acf9-72421ae759c5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:01:35.599007 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.598970    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4bed4b02-e289-4389-acf9-72421ae759c5" (UID: "4bed4b02-e289-4389-acf9-72421ae759c5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:35.604180 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.601499    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-config-volume" (OuterVolumeSpecName: "config-volume") pod "4bed4b02-e289-4389-acf9-72421ae759c5" (UID: "4bed4b02-e289-4389-acf9-72421ae759c5"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:35.607076 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.606298    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4bed4b02-e289-4389-acf9-72421ae759c5" (UID: "4bed4b02-e289-4389-acf9-72421ae759c5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:35.607076 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.606321    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4bed4b02-e289-4389-acf9-72421ae759c5" (UID: "4bed4b02-e289-4389-acf9-72421ae759c5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:35.607076 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.606416    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bed4b02-e289-4389-acf9-72421ae759c5-config-out" (OuterVolumeSpecName: "config-out") pod "4bed4b02-e289-4389-acf9-72421ae759c5" (UID: "4bed4b02-e289-4389-acf9-72421ae759c5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:01:35.608479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.608440    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4bed4b02-e289-4389-acf9-72421ae759c5" (UID: "4bed4b02-e289-4389-acf9-72421ae759c5"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:35.608479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.608458    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bed4b02-e289-4389-acf9-72421ae759c5-kube-api-access-wqh56" (OuterVolumeSpecName: "kube-api-access-wqh56") pod "4bed4b02-e289-4389-acf9-72421ae759c5" (UID: "4bed4b02-e289-4389-acf9-72421ae759c5"). InnerVolumeSpecName "kube-api-access-wqh56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:01:35.608720 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.608687    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4bed4b02-e289-4389-acf9-72421ae759c5" (UID: "4bed4b02-e289-4389-acf9-72421ae759c5"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:35.610211 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.610172    2578 generic.go:356] "Generic (PLEG): container finished" podID="4bed4b02-e289-4389-acf9-72421ae759c5" containerID="5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0" exitCode=0
Sep 06 05:01:35.610211 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.610210    2578 generic.go:356] "Generic (PLEG): container finished" podID="4bed4b02-e289-4389-acf9-72421ae759c5" containerID="4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa" exitCode=0
Sep 06 05:01:35.610350 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.610296    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerDied","Data":"5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0"}
Sep 06 05:01:35.610434 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.610409    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.610484 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.610452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerDied","Data":"4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa"}
Sep 06 05:01:35.610541 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.610497    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bed4b02-e289-4389-acf9-72421ae759c5","Type":"ContainerDied","Data":"de1a537c4d588a12e7cbd2a4d0dcb8a1c73deec9ac6e7c4ee80b890bcfe7a3c3"}
Sep 06 05:01:35.610589 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.610486    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-web-config" (OuterVolumeSpecName: "web-config") pod "4bed4b02-e289-4389-acf9-72421ae759c5" (UID: "4bed4b02-e289-4389-acf9-72421ae759c5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:35.610589 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.610527    2578 scope.go:117] "RemoveContainer" containerID="4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd"
Sep 06 05:01:35.619726 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.619706    2578 scope.go:117] "RemoveContainer" containerID="5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0"
Sep 06 05:01:35.627402 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.627386    2578 scope.go:117] "RemoveContainer" containerID="e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e"
Sep 06 05:01:35.634843 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.634817    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 06 05:01:35.635085 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.635065    2578 scope.go:117] "RemoveContainer" containerID="4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa"
Sep 06 05:01:35.637268 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.637247    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 06 05:01:35.642123 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.642104    2578 scope.go:117] "RemoveContainer" containerID="612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36"
Sep 06 05:01:35.649760 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.649741    2578 scope.go:117] "RemoveContainer" containerID="9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec"
Sep 06 05:01:35.656547 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.656529    2578 scope.go:117] "RemoveContainer" containerID="715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62"
Sep 06 05:01:35.660681 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.660658    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 06 05:01:35.660960 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.660944    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="init-config-reloader"
Sep 06 05:01:35.661046 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.660963    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="init-config-reloader"
Sep 06 05:01:35.661046 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.660976    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="prom-label-proxy"
Sep 06 05:01:35.661046 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.660985    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="prom-label-proxy"
Sep 06 05:01:35.661046 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.660999    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="kube-rbac-proxy"
Sep 06 05:01:35.661046 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661007    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="kube-rbac-proxy"
Sep 06 05:01:35.661293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661081    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="config-reloader"
Sep 06 05:01:35.661293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661092    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="config-reloader"
Sep 06 05:01:35.661293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661102    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="kube-rbac-proxy-web"
Sep 06 05:01:35.661293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661110    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="kube-rbac-proxy-web"
Sep 06 05:01:35.661293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661127    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="kube-rbac-proxy-metric"
Sep 06 05:01:35.661293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661136    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="kube-rbac-proxy-metric"
Sep 06 05:01:35.661293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661153    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="alertmanager"
Sep 06 05:01:35.661293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661161    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="alertmanager"
Sep 06 05:01:35.661293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661238    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="kube-rbac-proxy-web"
Sep 06 05:01:35.661293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661249    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="kube-rbac-proxy"
Sep 06 05:01:35.661293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661257    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="kube-rbac-proxy-metric"
Sep 06 05:01:35.661293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661267    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="config-reloader"
Sep 06 05:01:35.661293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661274    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="alertmanager"
Sep 06 05:01:35.661293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.661281    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" containerName="prom-label-proxy"
Sep 06 05:01:35.663320 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.663302    2578 scope.go:117] "RemoveContainer" containerID="4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd"
Sep 06 05:01:35.663590 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:01:35.663574    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd\": container with ID starting with 4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd not found: ID does not exist" containerID="4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd"
Sep 06 05:01:35.663630 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.663598    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd"} err="failed to get container status \"4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd\": rpc error: code = NotFound desc = could not find container \"4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd\": container with ID starting with 4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd not found: ID does not exist"
Sep 06 05:01:35.663630 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.663615    2578 scope.go:117] "RemoveContainer" containerID="5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0"
Sep 06 05:01:35.663854 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:01:35.663837    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0\": container with ID starting with 5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0 not found: ID does not exist" containerID="5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0"
Sep 06 05:01:35.663892 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.663862    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0"} err="failed to get container status \"5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0\": rpc error: code = NotFound desc = could not find container \"5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0\": container with ID starting with 5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0 not found: ID does not exist"
Sep 06 05:01:35.663892 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.663879    2578 scope.go:117] "RemoveContainer" containerID="e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e"
Sep 06 05:01:35.664103 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:01:35.664088    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e\": container with ID starting with e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e not found: ID does not exist" containerID="e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e"
Sep 06 05:01:35.664142 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.664108    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e"} err="failed to get container status \"e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e\": rpc error: code = NotFound desc = could not find container \"e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e\": container with ID starting with e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e not found: ID does not exist"
Sep 06 05:01:35.664142 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.664122    2578 scope.go:117] "RemoveContainer" containerID="4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa"
Sep 06 05:01:35.664357 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:01:35.664344    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa\": container with ID starting with 4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa not found: ID does not exist" containerID="4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa"
Sep 06 05:01:35.664394 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.664360    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa"} err="failed to get container status \"4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa\": rpc error: code = NotFound desc = could not find container \"4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa\": container with ID starting with 4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa not found: ID does not exist"
Sep 06 05:01:35.664394 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.664372    2578 scope.go:117] "RemoveContainer" containerID="612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36"
Sep 06 05:01:35.664559 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:01:35.664546    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36\": container with ID starting with 612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36 not found: ID does not exist" containerID="612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36"
Sep 06 05:01:35.664593 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.664563    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36"} err="failed to get container status \"612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36\": rpc error: code = NotFound desc = could not find container \"612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36\": container with ID starting with 612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36 not found: ID does not exist"
Sep 06 05:01:35.664593 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.664574    2578 scope.go:117] "RemoveContainer" containerID="9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec"
Sep 06 05:01:35.664778 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:01:35.664763    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec\": container with ID starting with 9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec not found: ID does not exist" containerID="9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec"
Sep 06 05:01:35.664812 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.664783    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec"} err="failed to get container status \"9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec\": rpc error: code = NotFound desc = could not find container \"9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec\": container with ID starting with 9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec not found: ID does not exist"
Sep 06 05:01:35.664812 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.664797    2578 scope.go:117] "RemoveContainer" containerID="715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62"
Sep 06 05:01:35.665046 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:01:35.665026    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62\": container with ID starting with 715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62 not found: ID does not exist" containerID="715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62"
Sep 06 05:01:35.665087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.665051    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62"} err="failed to get container status \"715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62\": rpc error: code = NotFound desc = could not find container \"715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62\": container with ID starting with 715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62 not found: ID does not exist"
Sep 06 05:01:35.665087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.665065    2578 scope.go:117] "RemoveContainer" containerID="4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd"
Sep 06 05:01:35.665307 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.665288    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd"} err="failed to get container status \"4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd\": rpc error: code = NotFound desc = could not find container \"4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd\": container with ID starting with 4717526282ab508a7e46403d6fdfedd318a4f86b6c65990feec0a2c0cf8a3cfd not found: ID does not exist"
Sep 06 05:01:35.665379 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.665309    2578 scope.go:117] "RemoveContainer" containerID="5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0"
Sep 06 05:01:35.665583 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.665566    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0"} err="failed to get container status \"5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0\": rpc error: code = NotFound desc = could not find container \"5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0\": container with ID starting with 5a99723205340e18cedacd8982d0b5aeff8f5e786521484d68737a464c10c6a0 not found: ID does not exist"
Sep 06 05:01:35.665631 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.665584    2578 scope.go:117] "RemoveContainer" containerID="e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e"
Sep 06 05:01:35.665774 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.665757    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e"} err="failed to get container status \"e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e\": rpc error: code = NotFound desc = could not find container \"e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e\": container with ID starting with e8ba8976d7da79ea20ad020d71961fd655fa7b75866cc27c4748669bf91c4a4e not found: ID does not exist"
Sep 06 05:01:35.665817 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.665773    2578 scope.go:117] "RemoveContainer" containerID="4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa"
Sep 06 05:01:35.665995 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.665974    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa"} err="failed to get container status \"4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa\": rpc error: code = NotFound desc = could not find container \"4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa\": container with ID starting with 4cb3d46630d76f6ad66d2f6adf27f28fa945889d29620d2c275c89cf231067aa not found: ID does not exist"
Sep 06 05:01:35.666088 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.665996    2578 scope.go:117] "RemoveContainer" containerID="612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36"
Sep 06 05:01:35.666285 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.666268    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36"} err="failed to get container status \"612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36\": rpc error: code = NotFound desc = could not find container \"612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36\": container with ID starting with 612e8e1a0f640e94e7cdd0e5f3d6fd35864177fc2e747c148149634ecce3ee36 not found: ID does not exist"
Sep 06 05:01:35.666336 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.666286    2578 scope.go:117] "RemoveContainer" containerID="9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec"
Sep 06 05:01:35.666512 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.666494    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec"} err="failed to get container status \"9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec\": rpc error: code = NotFound desc = could not find container \"9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec\": container with ID starting with 9eaeb6a2731ff95687e1077b31d02f30b71cc9a5c09c88d2ddb4f7537634deec not found: ID does not exist"
Sep 06 05:01:35.666557 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.666512    2578 scope.go:117] "RemoveContainer" containerID="715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62"
Sep 06 05:01:35.666725 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.666708    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62"} err="failed to get container status \"715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62\": rpc error: code = NotFound desc = could not find container \"715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62\": container with ID starting with 715b7d612fda4bcd6a611f9e8acceebcc76c75b5e5e66b11fcf946ce5bb40e62 not found: ID does not exist"
Sep 06 05:01:35.680732 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.680702    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 06 05:01:35.680849 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.680832    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.683136 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.683117    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 06 05:01:35.683303 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.683286    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9d8gk\""
Sep 06 05:01:35.683532 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.683516    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 06 05:01:35.683671 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.683644    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 06 05:01:35.683932 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.683910    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 06 05:01:35.683932 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.683924    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 06 05:01:35.684101 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.683929    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 06 05:01:35.684101 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.683963    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 06 05:01:35.684281 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.684260    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 06 05:01:35.687761 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.687738    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 06 05:01:35.689922 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.689901    2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4bed4b02-e289-4389-acf9-72421ae759c5-tls-assets\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:35.690045 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.689922    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:35.690045 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.689939    2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:35.690045 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.689953    2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4bed4b02-e289-4389-acf9-72421ae759c5-alertmanager-main-db\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:35.690045 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.689967    2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-cluster-tls-config\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:35.690045 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.689979    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:35.690045 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.689992    2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4bed4b02-e289-4389-acf9-72421ae759c5-config-out\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:35.690045 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.690005    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:35.690045 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.690037    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqh56\" (UniqueName: \"kubernetes.io/projected/4bed4b02-e289-4389-acf9-72421ae759c5-kube-api-access-wqh56\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:35.690297 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.690050    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-secret-alertmanager-main-tls\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:35.690297 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.690063    2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:35.690297 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.690075    2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4bed4b02-e289-4389-acf9-72421ae759c5-web-config\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:01:35.790728 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.790692    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.790728 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.790731    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.790992 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.790749    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b76fbf2-0a15-4385-8a20-7728b4d37269-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.790992 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.790769    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b76fbf2-0a15-4385-8a20-7728b4d37269-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.790992 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.790785    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqg6w\" (UniqueName: \"kubernetes.io/projected/4b76fbf2-0a15-4385-8a20-7728b4d37269-kube-api-access-cqg6w\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.790992 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.790838    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b76fbf2-0a15-4385-8a20-7728b4d37269-config-out\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.790992 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.790890    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-web-config\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.790992 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.790912    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.790992 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.790932    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.790992 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.790975    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.790992 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.790991    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b76fbf2-0a15-4385-8a20-7728b4d37269-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.791323 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.791008    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-config-volume\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.791323 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.791064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b76fbf2-0a15-4385-8a20-7728b4d37269-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.892098 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.892000    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b76fbf2-0a15-4385-8a20-7728b4d37269-config-out\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.892098 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.892069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-web-config\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.892098 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.892096    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.892399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.892116    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.892399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.892152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.892399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.892177    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b76fbf2-0a15-4385-8a20-7728b4d37269-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.892399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.892202    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-config-volume\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.892399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.892233    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b76fbf2-0a15-4385-8a20-7728b4d37269-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.892399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.892293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.892399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.892317    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.892399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.892343    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b76fbf2-0a15-4385-8a20-7728b4d37269-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.892399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.892377    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b76fbf2-0a15-4385-8a20-7728b4d37269-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.892399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.892401    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqg6w\" (UniqueName: \"kubernetes.io/projected/4b76fbf2-0a15-4385-8a20-7728b4d37269-kube-api-access-cqg6w\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.892876 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.892609    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b76fbf2-0a15-4385-8a20-7728b4d37269-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.893449 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.893421    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b76fbf2-0a15-4385-8a20-7728b4d37269-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.894245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.894216    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b76fbf2-0a15-4385-8a20-7728b4d37269-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.895114 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.895084    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b76fbf2-0a15-4385-8a20-7728b4d37269-config-out\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.895203 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.895175    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.895266 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.895250    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.895404 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.895367    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.895404 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.895367    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.895523 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.895456    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-config-volume\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.895566 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.895545    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-web-config\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.896334 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.896317    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b76fbf2-0a15-4385-8a20-7728b4d37269-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.896990 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.896974    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b76fbf2-0a15-4385-8a20-7728b4d37269-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.899413 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.899391    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqg6w\" (UniqueName: \"kubernetes.io/projected/4b76fbf2-0a15-4385-8a20-7728b4d37269-kube-api-access-cqg6w\") pod \"alertmanager-main-0\" (UID: \"4b76fbf2-0a15-4385-8a20-7728b4d37269\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:35.990134 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:35.990098    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 06 05:01:36.291165 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:36.291091    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 06 05:01:36.293793 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:01:36.293770    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b76fbf2_0a15_4385_8a20_7728b4d37269.slice/crio-a36f9738d16b9082097cd79349b22035d5acbc5a91f4d21542dba9e36c4c1428 WatchSource:0}: Error finding container a36f9738d16b9082097cd79349b22035d5acbc5a91f4d21542dba9e36c4c1428: Status 404 returned error can't find the container with id a36f9738d16b9082097cd79349b22035d5acbc5a91f4d21542dba9e36c4c1428
Sep 06 05:01:36.615727 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:36.615636    2578 generic.go:356] "Generic (PLEG): container finished" podID="4b76fbf2-0a15-4385-8a20-7728b4d37269" containerID="8661bed67198ed9d0cc1c344935324d96ebabe3f00819193887a7a3ad48f085b" exitCode=0
Sep 06 05:01:36.616212 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:36.615722    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b76fbf2-0a15-4385-8a20-7728b4d37269","Type":"ContainerDied","Data":"8661bed67198ed9d0cc1c344935324d96ebabe3f00819193887a7a3ad48f085b"}
Sep 06 05:01:36.616212 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:36.615816    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b76fbf2-0a15-4385-8a20-7728b4d37269","Type":"ContainerStarted","Data":"a36f9738d16b9082097cd79349b22035d5acbc5a91f4d21542dba9e36c4c1428"}
Sep 06 05:01:37.152565 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:37.152535    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bed4b02-e289-4389-acf9-72421ae759c5" path="/var/lib/kubelet/pods/4bed4b02-e289-4389-acf9-72421ae759c5/volumes"
Sep 06 05:01:37.623648 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:37.623620    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b76fbf2-0a15-4385-8a20-7728b4d37269","Type":"ContainerStarted","Data":"b5dce21d4be1bd0183e91496bd9f7a4c2010d2fe61ecbd2aa76963e0684acf2b"}
Sep 06 05:01:37.624009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:37.623683    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b76fbf2-0a15-4385-8a20-7728b4d37269","Type":"ContainerStarted","Data":"d9a8b21cfb02cafd496fc360656104b7c45c1134649e968c7222ba43f1b61b5c"}
Sep 06 05:01:37.624009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:37.623710    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b76fbf2-0a15-4385-8a20-7728b4d37269","Type":"ContainerStarted","Data":"2307fa545c5d46fc62cbf344dfc8bc5780d7d273ae0b8a5185bcb4a2dfe1b713"}
Sep 06 05:01:37.624009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:37.623737    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b76fbf2-0a15-4385-8a20-7728b4d37269","Type":"ContainerStarted","Data":"ebf2e63c3b14fe6ad7c4e1ef54571e0104c7a0d6c49638bad9accb0d9d95d7ae"}
Sep 06 05:01:37.624009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:37.623765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b76fbf2-0a15-4385-8a20-7728b4d37269","Type":"ContainerStarted","Data":"51b69e3c4574882e92901d8e714143bb5c6f50e661cda0ebb8b64863330966dc"}
Sep 06 05:01:37.624009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:37.623791    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b76fbf2-0a15-4385-8a20-7728b4d37269","Type":"ContainerStarted","Data":"7e80ddf7de1d0f90f017a064ad922764ababc4e37affd7394d9dbf8a322a6af0"}
Sep 06 05:01:37.655790 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:37.655756    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:37.655926 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:37.655790    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.6377868749999998 podStartE2EDuration="2.655754388s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.967513ms" isStatefulPod=false podCreationTimestamp="2026-09-06 05:01:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:01:37.655139893 +0000 UTC m=+365.024595139" watchObservedRunningTime="2026-09-06 05:01:37.655754388 +0000 UTC m=+365.025209635"
Sep 06 05:01:37.749096 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:37.749065    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:40.722278 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:40.722239    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:40.820804 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:40.820774    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:43.100334 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:43.100302    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:43.100729 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:43.100354    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:43.104900 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:43.104879    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:43.652620 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:43.652592    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:01:43.792355 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:43.792323    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:43.849240 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:43.849207    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:46.082830 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:46.082796    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 05:01:46.085092 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:46.085066    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d85cbcaa-00ae-4440-b7f0-7209f409a5cd-metrics-certs\") pod \"network-metrics-daemon-dxn9s\" (UID: \"d85cbcaa-00ae-4440-b7f0-7209f409a5cd\") " pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 05:01:46.251214 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:46.251184    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b9kvk\""
Sep 06 05:01:46.258841 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:46.258815    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxn9s"
Sep 06 05:01:46.524095 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:46.524071    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dxn9s"]
Sep 06 05:01:46.526065 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:01:46.526033    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd85cbcaa_00ae_4440_b7f0_7209f409a5cd.slice/crio-c58ea72253f03cb49f64d4e4ee8383a78788e5fc46d8d3d600d3bf3f59f224de WatchSource:0}: Error finding container c58ea72253f03cb49f64d4e4ee8383a78788e5fc46d8d3d600d3bf3f59f224de: Status 404 returned error can't find the container with id c58ea72253f03cb49f64d4e4ee8383a78788e5fc46d8d3d600d3bf3f59f224de
Sep 06 05:01:46.659284 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:46.659251    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dxn9s" event={"ID":"d85cbcaa-00ae-4440-b7f0-7209f409a5cd","Type":"ContainerStarted","Data":"c58ea72253f03cb49f64d4e4ee8383a78788e5fc46d8d3d600d3bf3f59f224de"}
Sep 06 05:01:47.666405 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:47.666357    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dxn9s" event={"ID":"d85cbcaa-00ae-4440-b7f0-7209f409a5cd","Type":"ContainerStarted","Data":"e23a7e81796d78c2018c1f80756c10a177def3e20f04acc52d4252fb73de732b"}
Sep 06 05:01:48.672163 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:48.672126    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dxn9s" event={"ID":"d85cbcaa-00ae-4440-b7f0-7209f409a5cd","Type":"ContainerStarted","Data":"edee513ec85bd6fc87cf43707f567d75e20d84259fc44b1f8efd23507485371c"}
Sep 06 05:01:48.687352 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:48.687279    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dxn9s" podStartSLOduration=374.663938259 podStartE2EDuration="6m15.687265043s" totalImagesPullingTime="1.023326784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:01:48.686555656 +0000 UTC m=+376.056010901" watchObservedRunningTime="2026-09-06 05:01:48.687265043 +0000 UTC m=+376.056720335"
Sep 06 05:01:48.766088 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:48.766054    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:48.815122 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:48.815089    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:48.873802 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:48.873768    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:54.848052 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:54.847999    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:01:54.898770 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:01:54.898740    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:12.682134 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:12.682099    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:12.779548 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:12.779516    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:31.803755 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:31.803706    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:31.868302 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:31.868272    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:39.244073 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:39.244008    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:40.002879 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:40.002843    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:42.842240 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:42.842205    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:42.891446 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:42.891413    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:44.315593 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:44.315557    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:44.452239 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:44.452208    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:45.161939 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:45.161908    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:45.266104 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:45.266075    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:45.710058 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:45.709999    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:45.766542 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:45.766513    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:47.179005 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:47.178974    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:47.588329 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:47.588249    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:49.644561 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:49.644526    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:49.699251 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:49.699216    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:54.277143 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:54.277102    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:54.373598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:54.373568    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:55.652743 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:55.652708    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:55.753732 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:55.753697    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:55.803412 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:55.803381    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:02:57.886396 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:57.886362    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6bj2f"]
Sep 06 05:02:57.889200 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:57.889184    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6bj2f"
Sep 06 05:02:57.891328 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:57.891304    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 06 05:02:57.892101 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:57.892079    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 06 05:02:57.892210 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:57.892106    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-gr8j7\""
Sep 06 05:02:57.897191 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:57.897169    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6bj2f"]
Sep 06 05:02:58.047673 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:58.047634    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx9s8\" (UniqueName: \"kubernetes.io/projected/0fef7531-decc-4a4d-ad24-c22669ee08e7-kube-api-access-rx9s8\") pod \"openshift-lws-operator-fd8ccff4c-6bj2f\" (UID: \"0fef7531-decc-4a4d-ad24-c22669ee08e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6bj2f"
Sep 06 05:02:58.047832 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:58.047682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fef7531-decc-4a4d-ad24-c22669ee08e7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6bj2f\" (UID: \"0fef7531-decc-4a4d-ad24-c22669ee08e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6bj2f"
Sep 06 05:02:58.148117 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:58.148000    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx9s8\" (UniqueName: \"kubernetes.io/projected/0fef7531-decc-4a4d-ad24-c22669ee08e7-kube-api-access-rx9s8\") pod \"openshift-lws-operator-fd8ccff4c-6bj2f\" (UID: \"0fef7531-decc-4a4d-ad24-c22669ee08e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6bj2f"
Sep 06 05:02:58.148117 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:58.148069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fef7531-decc-4a4d-ad24-c22669ee08e7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6bj2f\" (UID: \"0fef7531-decc-4a4d-ad24-c22669ee08e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6bj2f"
Sep 06 05:02:58.148382 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:58.148361    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fef7531-decc-4a4d-ad24-c22669ee08e7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6bj2f\" (UID: \"0fef7531-decc-4a4d-ad24-c22669ee08e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6bj2f"
Sep 06 05:02:58.156435 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:58.156398    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx9s8\" (UniqueName: \"kubernetes.io/projected/0fef7531-decc-4a4d-ad24-c22669ee08e7-kube-api-access-rx9s8\") pod \"openshift-lws-operator-fd8ccff4c-6bj2f\" (UID: \"0fef7531-decc-4a4d-ad24-c22669ee08e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6bj2f"
Sep 06 05:02:58.198540 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:58.198509    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6bj2f"
Sep 06 05:02:58.465939 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:58.465896    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6bj2f"]
Sep 06 05:02:58.468593 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:02:58.468566    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fef7531_decc_4a4d_ad24_c22669ee08e7.slice/crio-60716324895aa30d821e11efd7850e743446f253cc3c505b7915652a983e2d8c WatchSource:0}: Error finding container 60716324895aa30d821e11efd7850e743446f253cc3c505b7915652a983e2d8c: Status 404 returned error can't find the container with id 60716324895aa30d821e11efd7850e743446f253cc3c505b7915652a983e2d8c
Sep 06 05:02:58.929949 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:02:58.929917    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6bj2f" event={"ID":"0fef7531-decc-4a4d-ad24-c22669ee08e7","Type":"ContainerStarted","Data":"60716324895aa30d821e11efd7850e743446f253cc3c505b7915652a983e2d8c"}
Sep 06 05:03:01.943534 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:01.943498    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6bj2f" event={"ID":"0fef7531-decc-4a4d-ad24-c22669ee08e7","Type":"ContainerStarted","Data":"383e2f342dc43c821afa02f4dcd2be477b57504d50c96b069ba39f766ec00e5b"}
Sep 06 05:03:01.958455 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:01.958410    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6bj2f" podStartSLOduration=2.350013978 podStartE2EDuration="4.958394344s" totalImagesPullingTime="2.608380366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:02:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:03:01.956949635 +0000 UTC m=+449.326404884" watchObservedRunningTime="2026-09-06 05:03:01.958394344 +0000 UTC m=+449.327849589"
Sep 06 05:03:02.663056 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:02.663003    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:02.764334 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:02.764284    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:04.719790 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:04.719747    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:16.647896 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:16.647856    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:16.744119 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:16.744083    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:17.011820 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.011723    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:17.260399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.260358    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:17.368334 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.368309    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:17.389538 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.389498    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"]
Sep 06 05:03:17.392854 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.392830    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.395443 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.395395    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 06 05:03:17.395548 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.395446    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 06 05:03:17.395548 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.395395    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 06 05:03:17.395548 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.395414    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-p86vv\""
Sep 06 05:03:17.395677 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.395570    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 06 05:03:17.410527 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.410503    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"]
Sep 06 05:03:17.503290 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.503260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1cbba029-1e85-41bc-8342-44a5c5c33b7f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.503457 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.503300    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cbba029-1e85-41bc-8342-44a5c5c33b7f-webhook-cert\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.503457 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.503396    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1cbba029-1e85-41bc-8342-44a5c5c33b7f-manifests\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.503457 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.503444    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cbba029-1e85-41bc-8342-44a5c5c33b7f-tmp\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.503576 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.503474    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crvt8\" (UniqueName: \"kubernetes.io/projected/1cbba029-1e85-41bc-8342-44a5c5c33b7f-kube-api-access-crvt8\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.510044 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.509986    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:17.604128 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.604089    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cbba029-1e85-41bc-8342-44a5c5c33b7f-tmp\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.604311 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.604137    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crvt8\" (UniqueName: \"kubernetes.io/projected/1cbba029-1e85-41bc-8342-44a5c5c33b7f-kube-api-access-crvt8\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.604311 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.604170    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1cbba029-1e85-41bc-8342-44a5c5c33b7f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.604311 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.604187    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cbba029-1e85-41bc-8342-44a5c5c33b7f-webhook-cert\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.604311 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.604218    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1cbba029-1e85-41bc-8342-44a5c5c33b7f-manifests\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.604578 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.604556    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1cbba029-1e85-41bc-8342-44a5c5c33b7f-manifests\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.606392 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.606368    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cbba029-1e85-41bc-8342-44a5c5c33b7f-tmp\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.606622 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.606601    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cbba029-1e85-41bc-8342-44a5c5c33b7f-webhook-cert\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.606717 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.606697    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1cbba029-1e85-41bc-8342-44a5c5c33b7f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.613810 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.613789    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:17.615372 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.615352    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crvt8\" (UniqueName: \"kubernetes.io/projected/1cbba029-1e85-41bc-8342-44a5c5c33b7f-kube-api-access-crvt8\") pod \"opendatahub-operator-controller-manager-657c48fd97-wrlfn\" (UID: \"1cbba029-1e85-41bc-8342-44a5c5c33b7f\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.702857 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.702770    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:17.710548 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.710458    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:17.815439 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.815405    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:17.919642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.919612    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:17.976877 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:17.976808    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"]
Sep 06 05:03:17.979818 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:03:17.979785    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cbba029_1e85_41bc_8342_44a5c5c33b7f.slice/crio-70cfda08be53fc173d0f81cfffde1e76d64d201691d01f2aaf9675d35a6bae27 WatchSource:0}: Error finding container 70cfda08be53fc173d0f81cfffde1e76d64d201691d01f2aaf9675d35a6bae27: Status 404 returned error can't find the container with id 70cfda08be53fc173d0f81cfffde1e76d64d201691d01f2aaf9675d35a6bae27
Sep 06 05:03:18.004889 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:18.004862    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn" event={"ID":"1cbba029-1e85-41bc-8342-44a5c5c33b7f","Type":"ContainerStarted","Data":"70cfda08be53fc173d0f81cfffde1e76d64d201691d01f2aaf9675d35a6bae27"}
Sep 06 05:03:21.020623 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:21.020543    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn" event={"ID":"1cbba029-1e85-41bc-8342-44a5c5c33b7f","Type":"ContainerStarted","Data":"7fd22d2207c010f0acc30d00c464436335ff023a5e3876aca1fa2718f500e8c8"}
Sep 06 05:03:22.026302 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:22.026266    2578 generic.go:356] "Generic (PLEG): container finished" podID="1cbba029-1e85-41bc-8342-44a5c5c33b7f" containerID="7fd22d2207c010f0acc30d00c464436335ff023a5e3876aca1fa2718f500e8c8" exitCode=0
Sep 06 05:03:22.026662 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:22.026324    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn" event={"ID":"1cbba029-1e85-41bc-8342-44a5c5c33b7f","Type":"ContainerDied","Data":"7fd22d2207c010f0acc30d00c464436335ff023a5e3876aca1fa2718f500e8c8"}
Sep 06 05:03:23.032645 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:23.032604    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn" event={"ID":"1cbba029-1e85-41bc-8342-44a5c5c33b7f","Type":"ContainerStarted","Data":"782626ae5985655b453f058bdfd8183b1889c96dcd5bb344e95b79027321263f"}
Sep 06 05:03:23.033125 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:23.032706    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:23.051460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:23.051387    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn" podStartSLOduration=1.7592441650000001 podStartE2EDuration="6.051368597s" totalImagesPullingTime="3.222969457s" totalInitContainerRuntime="1.069154975s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:03:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:03:23.050451197 +0000 UTC m=+470.419906443" watchObservedRunningTime="2026-09-06 05:03:23.051368597 +0000 UTC m=+470.420823844"
Sep 06 05:03:24.162360 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:24.162289    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:24.226979 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:24.226943    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:33.192406 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:33.192374    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:33.292868 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:33.292833    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:34.038712 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:34.038681    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-wrlfn"
Sep 06 05:03:35.734540 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:35.734473    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:35.854899 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:35.854868    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:35.977752 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:35.977719    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:35.997057 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:35.996982    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6c96787df8-ndm87"]
Sep 06 05:03:36.088795 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.088760    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:36.091590 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.091570    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c96787df8-ndm87"]
Sep 06 05:03:36.091664 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.091654    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c96787df8-ndm87"
Sep 06 05:03:36.093883 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.093864    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 06 05:03:36.093883 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.093865    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 06 05:03:36.094928 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.094911    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 06 05:03:36.095011 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.094991    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 06 05:03:36.095011 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.094998    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-grg8c\""
Sep 06 05:03:36.141899 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.141864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbmsb\" (UniqueName: \"kubernetes.io/projected/23793d3c-462d-4a32-a0aa-56cd314392a8-kube-api-access-tbmsb\") pod \"kube-auth-proxy-6c96787df8-ndm87\" (UID: \"23793d3c-462d-4a32-a0aa-56cd314392a8\") " pod="openshift-ingress/kube-auth-proxy-6c96787df8-ndm87"
Sep 06 05:03:36.142085 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.141926    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23793d3c-462d-4a32-a0aa-56cd314392a8-tmp\") pod \"kube-auth-proxy-6c96787df8-ndm87\" (UID: \"23793d3c-462d-4a32-a0aa-56cd314392a8\") " pod="openshift-ingress/kube-auth-proxy-6c96787df8-ndm87"
Sep 06 05:03:36.142085 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.141955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23793d3c-462d-4a32-a0aa-56cd314392a8-tls-certs\") pod \"kube-auth-proxy-6c96787df8-ndm87\" (UID: \"23793d3c-462d-4a32-a0aa-56cd314392a8\") " pod="openshift-ingress/kube-auth-proxy-6c96787df8-ndm87"
Sep 06 05:03:36.242535 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.242495    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbmsb\" (UniqueName: \"kubernetes.io/projected/23793d3c-462d-4a32-a0aa-56cd314392a8-kube-api-access-tbmsb\") pod \"kube-auth-proxy-6c96787df8-ndm87\" (UID: \"23793d3c-462d-4a32-a0aa-56cd314392a8\") " pod="openshift-ingress/kube-auth-proxy-6c96787df8-ndm87"
Sep 06 05:03:36.242683 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.242547    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23793d3c-462d-4a32-a0aa-56cd314392a8-tmp\") pod \"kube-auth-proxy-6c96787df8-ndm87\" (UID: \"23793d3c-462d-4a32-a0aa-56cd314392a8\") " pod="openshift-ingress/kube-auth-proxy-6c96787df8-ndm87"
Sep 06 05:03:36.242683 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.242581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23793d3c-462d-4a32-a0aa-56cd314392a8-tls-certs\") pod \"kube-auth-proxy-6c96787df8-ndm87\" (UID: \"23793d3c-462d-4a32-a0aa-56cd314392a8\") " pod="openshift-ingress/kube-auth-proxy-6c96787df8-ndm87"
Sep 06 05:03:36.244863 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.244838    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23793d3c-462d-4a32-a0aa-56cd314392a8-tmp\") pod \"kube-auth-proxy-6c96787df8-ndm87\" (UID: \"23793d3c-462d-4a32-a0aa-56cd314392a8\") " pod="openshift-ingress/kube-auth-proxy-6c96787df8-ndm87"
Sep 06 05:03:36.244980 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.244933    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23793d3c-462d-4a32-a0aa-56cd314392a8-tls-certs\") pod \"kube-auth-proxy-6c96787df8-ndm87\" (UID: \"23793d3c-462d-4a32-a0aa-56cd314392a8\") " pod="openshift-ingress/kube-auth-proxy-6c96787df8-ndm87"
Sep 06 05:03:36.252382 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.252326    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbmsb\" (UniqueName: \"kubernetes.io/projected/23793d3c-462d-4a32-a0aa-56cd314392a8-kube-api-access-tbmsb\") pod \"kube-auth-proxy-6c96787df8-ndm87\" (UID: \"23793d3c-462d-4a32-a0aa-56cd314392a8\") " pod="openshift-ingress/kube-auth-proxy-6c96787df8-ndm87"
Sep 06 05:03:36.400785 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.400733    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c96787df8-ndm87"
Sep 06 05:03:36.691496 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:36.691464    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c96787df8-ndm87"]
Sep 06 05:03:36.694230 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:03:36.694206    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23793d3c_462d_4a32_a0aa_56cd314392a8.slice/crio-6ff84fd53deadd11e0402c916cf1f04eadae2fa45b2be017b87c6b24b477cc30 WatchSource:0}: Error finding container 6ff84fd53deadd11e0402c916cf1f04eadae2fa45b2be017b87c6b24b477cc30: Status 404 returned error can't find the container with id 6ff84fd53deadd11e0402c916cf1f04eadae2fa45b2be017b87c6b24b477cc30
Sep 06 05:03:37.089239 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.089102    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c96787df8-ndm87" event={"ID":"23793d3c-462d-4a32-a0aa-56cd314392a8","Type":"ContainerStarted","Data":"6ff84fd53deadd11e0402c916cf1f04eadae2fa45b2be017b87c6b24b477cc30"}
Sep 06 05:03:37.168302 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.168268    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"]
Sep 06 05:03:37.172191 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.172165    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:37.198748 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.198720    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"]
Sep 06 05:03:37.198878 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.198821    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.201111 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.201085    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 06 05:03:37.201244 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.201128    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 06 05:03:37.201244 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.201231    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 06 05:03:37.201244 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.201237    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 06 05:03:37.201379 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.201354    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qpw86\""
Sep 06 05:03:37.233444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.233414    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:37.250608 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.250586    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9ef163-771b-443a-b18a-8f62f7304430-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.250732 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.250629    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.250794 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.250727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff9ef163-771b-443a-b18a-8f62f7304430-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.250794 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.250776    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.250893 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.250832    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.250893 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.250858    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmndd\" (UniqueName: \"kubernetes.io/projected/ff9ef163-771b-443a-b18a-8f62f7304430-kube-api-access-nmndd\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.250893 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.250885    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff9ef163-771b-443a-b18a-8f62f7304430-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.352259 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.352169    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.352259 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.352216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff9ef163-771b-443a-b18a-8f62f7304430-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.352259 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.352240    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.352633 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.352273    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.352633 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.352294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmndd\" (UniqueName: \"kubernetes.io/projected/ff9ef163-771b-443a-b18a-8f62f7304430-kube-api-access-nmndd\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.352633 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.352311    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff9ef163-771b-443a-b18a-8f62f7304430-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.352781 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.352654    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9ef163-771b-443a-b18a-8f62f7304430-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.352941 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.352916    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff9ef163-771b-443a-b18a-8f62f7304430-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.355081 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.355058    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.355288 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.355271    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9ef163-771b-443a-b18a-8f62f7304430-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.355453 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.355410    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.355542 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.355483    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.360161 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.360139    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff9ef163-771b-443a-b18a-8f62f7304430-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.360641 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.360619    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmndd\" (UniqueName: \"kubernetes.io/projected/ff9ef163-771b-443a-b18a-8f62f7304430-kube-api-access-nmndd\") pod \"istiod-openshift-gateway-7b7995d5cd-8zdt9\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.508229 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.508193    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:37.796529 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:37.796494    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"]
Sep 06 05:03:37.800115 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:03:37.800085    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff9ef163_771b_443a_b18a_8f62f7304430.slice/crio-c19cb6458d356b611b0d4a1eca2080245ab88049dda088ff6c33402fd2b4dd7f WatchSource:0}: Error finding container c19cb6458d356b611b0d4a1eca2080245ab88049dda088ff6c33402fd2b4dd7f: Status 404 returned error can't find the container with id c19cb6458d356b611b0d4a1eca2080245ab88049dda088ff6c33402fd2b4dd7f
Sep 06 05:03:38.097373 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:38.097275    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9" event={"ID":"ff9ef163-771b-443a-b18a-8f62f7304430","Type":"ContainerStarted","Data":"c19cb6458d356b611b0d4a1eca2080245ab88049dda088ff6c33402fd2b4dd7f"}
Sep 06 05:03:38.186134 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:38.186100    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:40.109160 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:40.109119    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c96787df8-ndm87" event={"ID":"23793d3c-462d-4a32-a0aa-56cd314392a8","Type":"ContainerStarted","Data":"6b1516a21b5ffaacd28157aaea7674dd252018c5d13aac33d16905fc3176d70a"}
Sep 06 05:03:40.125714 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:40.125655    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6c96787df8-ndm87" podStartSLOduration=2.422479472 podStartE2EDuration="5.125639295s" totalImagesPullingTime="2.703159823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:03:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:03:40.122917414 +0000 UTC m=+487.492372661" watchObservedRunningTime="2026-09-06 05:03:40.125639295 +0000 UTC m=+487.495094540"
Sep 06 05:03:41.078627 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.078585    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:03:41.078743 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.078666    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:03:41.432688 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.432646    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:41.530322 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.530264    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:41.618218 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.618176    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:41.893818 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.893783    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"]
Sep 06 05:03:41.895945 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.895927    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:41.898919 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.898895    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-lzdvg\""
Sep 06 05:03:41.916762 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.916731    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"]
Sep 06 05:03:41.928169 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.928141    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:41.996033 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.995969    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:41.996033 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.996010    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:41.996256 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.996081    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:41.996256 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.996118    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:41.996256 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.996156    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:41.996256 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.996180    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:41.996449 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.996263    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:41.996449 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.996321    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:41.996449 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:41.996351    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgj4q\" (UniqueName: \"kubernetes.io/projected/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-kube-api-access-dgj4q\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.009296 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.009266    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:42.097169 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.097128    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.097169 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.097174    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.097391 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.097215    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.097391 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.097233    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.097391 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.097272    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.097391 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.097304    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.097391 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.097328    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgj4q\" (UniqueName: \"kubernetes.io/projected/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-kube-api-access-dgj4q\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.097391 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.097386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.097682 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.097413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.097682 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.097567    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.097682 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.097646    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.097809 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.097703    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.097809 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.097760    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.098070 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.098054    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.099692 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.099670    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.099881 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.099864    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.105148 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.105127    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.105376 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.105344    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgj4q\" (UniqueName: \"kubernetes.io/projected/9a81cc22-70c0-49e9-a55a-9bfa627dcf5f-kube-api-access-dgj4q\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf\" (UID: \"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.119589 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.119558    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9" event={"ID":"ff9ef163-771b-443a-b18a-8f62f7304430","Type":"ContainerStarted","Data":"01f65af2b6a672313177891e6a3c5898cb2b8b20a319168b742aa98b0bcd1738"}
Sep 06 05:03:42.119703 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.119673    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:42.121389 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.121362    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:03:42.139755 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.139706    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9" podStartSLOduration=1.864285755 podStartE2EDuration="5.13969462s" totalImagesPullingTime="3.275408865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:03:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:03:42.138062695 +0000 UTC m=+489.507517941" watchObservedRunningTime="2026-09-06 05:03:42.13969462 +0000 UTC m=+489.509149865"
Sep 06 05:03:42.205898 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.205829    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:42.477112 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:42.477026    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"]
Sep 06 05:03:42.479810 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:03:42.479778    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a81cc22_70c0_49e9_a55a_9bfa627dcf5f.slice/crio-4eae808677edae8482c56c4ddddab93a598ff982fcd53b5540c65f6ea7b8e8dd WatchSource:0}: Error finding container 4eae808677edae8482c56c4ddddab93a598ff982fcd53b5540c65f6ea7b8e8dd: Status 404 returned error can't find the container with id 4eae808677edae8482c56c4ddddab93a598ff982fcd53b5540c65f6ea7b8e8dd
Sep 06 05:03:43.124919 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:43.124880    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf" event={"ID":"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f","Type":"ContainerStarted","Data":"4eae808677edae8482c56c4ddddab93a598ff982fcd53b5540c65f6ea7b8e8dd"}
Sep 06 05:03:46.542122 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:46.542077    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:03:46.542403 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:46.542151    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:03:46.542403 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:46.542181    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:03:47.096075 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:47.096043    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:47.144999 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:47.144965    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf" event={"ID":"9a81cc22-70c0-49e9-a55a-9bfa627dcf5f","Type":"ContainerStarted","Data":"161c3ca7e6827f4b23902d0afcf0782fee62998dbc34e6ea88865a75e1a7e1ee"}
Sep 06 05:03:47.149985 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:47.149957    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:47.163224 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:47.163175    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf" podStartSLOduration=2.103091917 podStartE2EDuration="6.163160613s" totalImagesPullingTime="4.060068696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:03:47.162210399 +0000 UTC m=+494.531665678" watchObservedRunningTime="2026-09-06 05:03:47.163160613 +0000 UTC m=+494.532615860"
Sep 06 05:03:47.206369 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:47.206326    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:47.208207 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:47.208181    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:47.211237 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:47.211218    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:47.260068 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:47.260036    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:47.443904 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:47.443869    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:47.582585 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:47.582539    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:48.109040 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:48.108991    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:48.148825 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:48.148789    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:48.149904 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:48.149886    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf"
Sep 06 05:03:49.760056 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:49.760003    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:49.814129 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:49.814094    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:56.526388 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.526355    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:56.569579 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.569547    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gtn45"]
Sep 06 05:03:56.571716 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.571695    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gtn45"
Sep 06 05:03:56.573967 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.573948    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 06 05:03:56.574103 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.573952    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-wkjfc\""
Sep 06 05:03:56.580219 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.580198    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4"]
Sep 06 05:03:56.582807 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.582789    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4"
Sep 06 05:03:56.582922 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.582905    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gtn45"]
Sep 06 05:03:56.585238 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.585217    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-n5f7h\""
Sep 06 05:03:56.585336 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.585262    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 06 05:03:56.605302 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.605277    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4"]
Sep 06 05:03:56.608183 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.608159    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:56.617316 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.617293    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b42b54e-dd87-427f-87e7-3620664c6f4e-cert\") pod \"kserve-controller-manager-d6f4b65b8-gtn45\" (UID: \"0b42b54e-dd87-427f-87e7-3620664c6f4e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gtn45"
Sep 06 05:03:56.617415 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.617334    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rr9g\" (UniqueName: \"kubernetes.io/projected/0b42b54e-dd87-427f-87e7-3620664c6f4e-kube-api-access-2rr9g\") pod \"kserve-controller-manager-d6f4b65b8-gtn45\" (UID: \"0b42b54e-dd87-427f-87e7-3620664c6f4e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gtn45"
Sep 06 05:03:56.718454 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.718414    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cad8cce1-0dec-42c9-9b98-d706cc3ec37c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-g2nw4\" (UID: \"cad8cce1-0dec-42c9-9b98-d706cc3ec37c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4"
Sep 06 05:03:56.718454 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.718461    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b42b54e-dd87-427f-87e7-3620664c6f4e-cert\") pod \"kserve-controller-manager-d6f4b65b8-gtn45\" (UID: \"0b42b54e-dd87-427f-87e7-3620664c6f4e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gtn45"
Sep 06 05:03:56.718713 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.718499    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rr9g\" (UniqueName: \"kubernetes.io/projected/0b42b54e-dd87-427f-87e7-3620664c6f4e-kube-api-access-2rr9g\") pod \"kserve-controller-manager-d6f4b65b8-gtn45\" (UID: \"0b42b54e-dd87-427f-87e7-3620664c6f4e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gtn45"
Sep 06 05:03:56.718713 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.718517    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-272jd\" (UniqueName: \"kubernetes.io/projected/cad8cce1-0dec-42c9-9b98-d706cc3ec37c-kube-api-access-272jd\") pod \"llmisvc-controller-manager-777c6bd7d5-g2nw4\" (UID: \"cad8cce1-0dec-42c9-9b98-d706cc3ec37c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4"
Sep 06 05:03:56.718713 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:03:56.718582    2578 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 06 05:03:56.718713 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:03:56.718661    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b42b54e-dd87-427f-87e7-3620664c6f4e-cert podName:0b42b54e-dd87-427f-87e7-3620664c6f4e nodeName:}" failed. No retries permitted until 2026-09-06 05:03:57.218644319 +0000 UTC m=+504.588099543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0b42b54e-dd87-427f-87e7-3620664c6f4e-cert") pod "kserve-controller-manager-d6f4b65b8-gtn45" (UID: "0b42b54e-dd87-427f-87e7-3620664c6f4e") : secret "kserve-webhook-server-cert" not found
Sep 06 05:03:56.730127 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.730100    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rr9g\" (UniqueName: \"kubernetes.io/projected/0b42b54e-dd87-427f-87e7-3620664c6f4e-kube-api-access-2rr9g\") pod \"kserve-controller-manager-d6f4b65b8-gtn45\" (UID: \"0b42b54e-dd87-427f-87e7-3620664c6f4e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gtn45"
Sep 06 05:03:56.819286 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.819193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cad8cce1-0dec-42c9-9b98-d706cc3ec37c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-g2nw4\" (UID: \"cad8cce1-0dec-42c9-9b98-d706cc3ec37c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4"
Sep 06 05:03:56.819426 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.819292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-272jd\" (UniqueName: \"kubernetes.io/projected/cad8cce1-0dec-42c9-9b98-d706cc3ec37c-kube-api-access-272jd\") pod \"llmisvc-controller-manager-777c6bd7d5-g2nw4\" (UID: \"cad8cce1-0dec-42c9-9b98-d706cc3ec37c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4"
Sep 06 05:03:56.819426 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:03:56.819361    2578 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 06 05:03:56.819538 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:03:56.819438    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cad8cce1-0dec-42c9-9b98-d706cc3ec37c-cert podName:cad8cce1-0dec-42c9-9b98-d706cc3ec37c nodeName:}" failed. No retries permitted until 2026-09-06 05:03:57.319421237 +0000 UTC m=+504.688876484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cad8cce1-0dec-42c9-9b98-d706cc3ec37c-cert") pod "llmisvc-controller-manager-777c6bd7d5-g2nw4" (UID: "cad8cce1-0dec-42c9-9b98-d706cc3ec37c") : secret "llmisvc-webhook-server-cert" not found
Sep 06 05:03:56.841234 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:56.841205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-272jd\" (UniqueName: \"kubernetes.io/projected/cad8cce1-0dec-42c9-9b98-d706cc3ec37c-kube-api-access-272jd\") pod \"llmisvc-controller-manager-777c6bd7d5-g2nw4\" (UID: \"cad8cce1-0dec-42c9-9b98-d706cc3ec37c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4"
Sep 06 05:03:57.160567 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:57.160533    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:57.210603 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:57.210571    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:57.223232 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:57.223207    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b42b54e-dd87-427f-87e7-3620664c6f4e-cert\") pod \"kserve-controller-manager-d6f4b65b8-gtn45\" (UID: \"0b42b54e-dd87-427f-87e7-3620664c6f4e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gtn45"
Sep 06 05:03:57.225926 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:57.225898    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b42b54e-dd87-427f-87e7-3620664c6f4e-cert\") pod \"kserve-controller-manager-d6f4b65b8-gtn45\" (UID: \"0b42b54e-dd87-427f-87e7-3620664c6f4e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gtn45"
Sep 06 05:03:57.324468 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:57.324437    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cad8cce1-0dec-42c9-9b98-d706cc3ec37c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-g2nw4\" (UID: \"cad8cce1-0dec-42c9-9b98-d706cc3ec37c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4"
Sep 06 05:03:57.326743 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:57.326716    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cad8cce1-0dec-42c9-9b98-d706cc3ec37c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-g2nw4\" (UID: \"cad8cce1-0dec-42c9-9b98-d706cc3ec37c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4"
Sep 06 05:03:57.482458 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:57.482377    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gtn45"
Sep 06 05:03:57.492296 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:57.492270    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4"
Sep 06 05:03:57.778098 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:57.778074    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gtn45"]
Sep 06 05:03:57.779470 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:57.779445    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4"]
Sep 06 05:03:57.779872 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:03:57.779849    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcad8cce1_0dec_42c9_9b98_d706cc3ec37c.slice/crio-7c29c29561b690788cfc6f48e7d54437a56331a0ac3af1a1986fc3e52485857e WatchSource:0}: Error finding container 7c29c29561b690788cfc6f48e7d54437a56331a0ac3af1a1986fc3e52485857e: Status 404 returned error can't find the container with id 7c29c29561b690788cfc6f48e7d54437a56331a0ac3af1a1986fc3e52485857e
Sep 06 05:03:57.780437 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:03:57.780421    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b42b54e_dd87_427f_87e7_3620664c6f4e.slice/crio-6f82aa2b7fdddea9eab4f84fbf5d0e3fe5f94474c31dc639fb3be6461071275c WatchSource:0}: Error finding container 6f82aa2b7fdddea9eab4f84fbf5d0e3fe5f94474c31dc639fb3be6461071275c: Status 404 returned error can't find the container with id 6f82aa2b7fdddea9eab4f84fbf5d0e3fe5f94474c31dc639fb3be6461071275c
Sep 06 05:03:57.825912 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:57.825879    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:57.927973 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:57.927943    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:03:58.189712 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:58.189665    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4" event={"ID":"cad8cce1-0dec-42c9-9b98-d706cc3ec37c","Type":"ContainerStarted","Data":"7c29c29561b690788cfc6f48e7d54437a56331a0ac3af1a1986fc3e52485857e"}
Sep 06 05:03:58.190872 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:58.190848    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gtn45" event={"ID":"0b42b54e-dd87-427f-87e7-3620664c6f4e","Type":"ContainerStarted","Data":"6f82aa2b7fdddea9eab4f84fbf5d0e3fe5f94474c31dc639fb3be6461071275c"}
Sep 06 05:03:59.313093 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:59.312960    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"]
Sep 06 05:03:59.313675 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:03:59.313273    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9" podUID="ff9ef163-771b-443a-b18a-8f62f7304430" containerName="discovery" containerID="cri-o://01f65af2b6a672313177891e6a3c5898cb2b8b20a319168b742aa98b0bcd1738" gracePeriod=30
Sep 06 05:04:00.152501 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.152473    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:04:00.205695 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.205669    2578 generic.go:356] "Generic (PLEG): container finished" podID="ff9ef163-771b-443a-b18a-8f62f7304430" containerID="01f65af2b6a672313177891e6a3c5898cb2b8b20a319168b742aa98b0bcd1738" exitCode=0
Sep 06 05:04:00.205823 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.205723    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"
Sep 06 05:04:00.205823 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.205766    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9" event={"ID":"ff9ef163-771b-443a-b18a-8f62f7304430","Type":"ContainerDied","Data":"01f65af2b6a672313177891e6a3c5898cb2b8b20a319168b742aa98b0bcd1738"}
Sep 06 05:04:00.205943 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.205833    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9" event={"ID":"ff9ef163-771b-443a-b18a-8f62f7304430","Type":"ContainerDied","Data":"c19cb6458d356b611b0d4a1eca2080245ab88049dda088ff6c33402fd2b4dd7f"}
Sep 06 05:04:00.205943 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.205865    2578 scope.go:117] "RemoveContainer" containerID="01f65af2b6a672313177891e6a3c5898cb2b8b20a319168b742aa98b0bcd1738"
Sep 06 05:04:00.229460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.229441    2578 scope.go:117] "RemoveContainer" containerID="01f65af2b6a672313177891e6a3c5898cb2b8b20a319168b742aa98b0bcd1738"
Sep 06 05:04:00.229778 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:04:00.229758    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01f65af2b6a672313177891e6a3c5898cb2b8b20a319168b742aa98b0bcd1738\": container with ID starting with 01f65af2b6a672313177891e6a3c5898cb2b8b20a319168b742aa98b0bcd1738 not found: ID does not exist" containerID="01f65af2b6a672313177891e6a3c5898cb2b8b20a319168b742aa98b0bcd1738"
Sep 06 05:04:00.229870 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.229790    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01f65af2b6a672313177891e6a3c5898cb2b8b20a319168b742aa98b0bcd1738"} err="failed to get container status \"01f65af2b6a672313177891e6a3c5898cb2b8b20a319168b742aa98b0bcd1738\": rpc error: code = NotFound desc = could not find container \"01f65af2b6a672313177891e6a3c5898cb2b8b20a319168b742aa98b0bcd1738\": container with ID starting with 01f65af2b6a672313177891e6a3c5898cb2b8b20a319168b742aa98b0bcd1738 not found: ID does not exist"
Sep 06 05:04:00.253001 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.252978    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmndd\" (UniqueName: \"kubernetes.io/projected/ff9ef163-771b-443a-b18a-8f62f7304430-kube-api-access-nmndd\") pod \"ff9ef163-771b-443a-b18a-8f62f7304430\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") "
Sep 06 05:04:00.253132 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.253034    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-istio-csr-dns-cert\") pod \"ff9ef163-771b-443a-b18a-8f62f7304430\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") "
Sep 06 05:04:00.253132 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.253061    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff9ef163-771b-443a-b18a-8f62f7304430-istio-token\") pod \"ff9ef163-771b-443a-b18a-8f62f7304430\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") "
Sep 06 05:04:00.253132 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.253109    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-cacerts\") pod \"ff9ef163-771b-443a-b18a-8f62f7304430\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") "
Sep 06 05:04:00.253298 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.253136    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff9ef163-771b-443a-b18a-8f62f7304430-istio-csr-ca-configmap\") pod \"ff9ef163-771b-443a-b18a-8f62f7304430\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") "
Sep 06 05:04:00.253298 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.253164    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-istio-kubeconfig\") pod \"ff9ef163-771b-443a-b18a-8f62f7304430\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") "
Sep 06 05:04:00.253298 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.253190    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9ef163-771b-443a-b18a-8f62f7304430-local-certs\") pod \"ff9ef163-771b-443a-b18a-8f62f7304430\" (UID: \"ff9ef163-771b-443a-b18a-8f62f7304430\") "
Sep 06 05:04:00.253837 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.253758    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff9ef163-771b-443a-b18a-8f62f7304430-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "ff9ef163-771b-443a-b18a-8f62f7304430" (UID: "ff9ef163-771b-443a-b18a-8f62f7304430"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:04:00.255757 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.255705    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9ef163-771b-443a-b18a-8f62f7304430-local-certs" (OuterVolumeSpecName: "local-certs") pod "ff9ef163-771b-443a-b18a-8f62f7304430" (UID: "ff9ef163-771b-443a-b18a-8f62f7304430"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:04:00.256052 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.256007    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "ff9ef163-771b-443a-b18a-8f62f7304430" (UID: "ff9ef163-771b-443a-b18a-8f62f7304430"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:04:00.256126 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.256029    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9ef163-771b-443a-b18a-8f62f7304430-kube-api-access-nmndd" (OuterVolumeSpecName: "kube-api-access-nmndd") pod "ff9ef163-771b-443a-b18a-8f62f7304430" (UID: "ff9ef163-771b-443a-b18a-8f62f7304430"). InnerVolumeSpecName "kube-api-access-nmndd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:04:00.256499 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.256463    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "ff9ef163-771b-443a-b18a-8f62f7304430" (UID: "ff9ef163-771b-443a-b18a-8f62f7304430"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:04:00.256599 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.256543    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-cacerts" (OuterVolumeSpecName: "cacerts") pod "ff9ef163-771b-443a-b18a-8f62f7304430" (UID: "ff9ef163-771b-443a-b18a-8f62f7304430"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:04:00.257144 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.257118    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9ef163-771b-443a-b18a-8f62f7304430-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff9ef163-771b-443a-b18a-8f62f7304430" (UID: "ff9ef163-771b-443a-b18a-8f62f7304430"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:04:00.354127 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.354093    2578 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-cacerts\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:04:00.354127 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.354120    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff9ef163-771b-443a-b18a-8f62f7304430-istio-csr-ca-configmap\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:04:00.354127 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.354132    2578 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-istio-kubeconfig\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:04:00.354689 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.354146    2578 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9ef163-771b-443a-b18a-8f62f7304430-local-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:04:00.354689 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.354158    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmndd\" (UniqueName: \"kubernetes.io/projected/ff9ef163-771b-443a-b18a-8f62f7304430-kube-api-access-nmndd\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:04:00.354689 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.354173    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff9ef163-771b-443a-b18a-8f62f7304430-istio-csr-dns-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:04:00.354689 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.354189    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff9ef163-771b-443a-b18a-8f62f7304430-istio-token\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:04:00.529441 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.529414    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"]
Sep 06 05:04:00.533167 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:00.533138    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9"]
Sep 06 05:04:01.154464 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:01.154433    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff9ef163-771b-443a-b18a-8f62f7304430" path="/var/lib/kubelet/pods/ff9ef163-771b-443a-b18a-8f62f7304430/volumes"
Sep 06 05:04:01.217962 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:01.217924    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gtn45" event={"ID":"0b42b54e-dd87-427f-87e7-3620664c6f4e","Type":"ContainerStarted","Data":"e58ecbb9995eff48b6f353beee4ef2236d7c15bea9312b8faf5a5e7af9463b56"}
Sep 06 05:04:01.218208 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:01.218068    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gtn45"
Sep 06 05:04:01.219621 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:01.219594    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4" event={"ID":"cad8cce1-0dec-42c9-9b98-d706cc3ec37c","Type":"ContainerStarted","Data":"2238fed15dfe7450d779f747f02290947c69bbe9e632d6ae3efe55f8b9ca7a75"}
Sep 06 05:04:01.219719 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:01.219649    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4"
Sep 06 05:04:01.233585 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:01.233544    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gtn45" podStartSLOduration=2.830465889 podStartE2EDuration="5.233530613s" totalImagesPullingTime="2.403064724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:03:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:04:01.233037124 +0000 UTC m=+508.602492389" watchObservedRunningTime="2026-09-06 05:04:01.233530613 +0000 UTC m=+508.602985856"
Sep 06 05:04:01.248149 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:01.248108    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4" podStartSLOduration=2.185923151 podStartE2EDuration="5.248096356s" totalImagesPullingTime="3.062173205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:03:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:04:01.24755361 +0000 UTC m=+508.617008858" watchObservedRunningTime="2026-09-06 05:04:01.248096356 +0000 UTC m=+508.617551605"
Sep 06 05:04:05.120140 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:05.120083    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-8zdt9 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.21:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 05:04:05.120540 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:05.120159    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8zdt9" podUID="ff9ef163-771b-443a-b18a-8f62f7304430" containerName="discovery" probeResult="failure" output="Get \"http://10.133.0.21:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 05:04:32.227957 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:32.227879    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g2nw4"
Sep 06 05:04:32.230915 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:32.230897    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gtn45"
Sep 06 05:04:48.055918 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:48.055865    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:04:49.507770 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:04:49.507726    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:00.981687 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:00.981649    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:21.638008 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:21.637964    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:22.429599 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:22.429555    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:23.230363 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:23.230323    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:25.953066 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:25.953033    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"]
Sep 06 05:05:25.953550 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:25.953515    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff9ef163-771b-443a-b18a-8f62f7304430" containerName="discovery"
Sep 06 05:05:25.953550 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:25.953533    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff9ef163-771b-443a-b18a-8f62f7304430" containerName="discovery"
Sep 06 05:05:25.953659 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:25.953605    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff9ef163-771b-443a-b18a-8f62f7304430" containerName="discovery"
Sep 06 05:05:25.956642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:25.956622    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:25.959053 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:25.959032    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bs2v2\""
Sep 06 05:05:25.959156 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:25.959082    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 06 05:05:25.959221 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:25.959205    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 06 05:05:25.969942 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:25.969919    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"]
Sep 06 05:05:25.970413 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:25.970391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92s8g\" (UniqueName: \"kubernetes.io/projected/81921157-9b41-4137-9f42-5c4e34307143-kube-api-access-92s8g\") pod \"kuadrant-operator-controller-manager-58fc56757c-lrr9f\" (UID: \"81921157-9b41-4137-9f42-5c4e34307143\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:25.970468 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:25.970428    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81921157-9b41-4137-9f42-5c4e34307143-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-lrr9f\" (UID: \"81921157-9b41-4137-9f42-5c4e34307143\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:26.071488 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:26.071450    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92s8g\" (UniqueName: \"kubernetes.io/projected/81921157-9b41-4137-9f42-5c4e34307143-kube-api-access-92s8g\") pod \"kuadrant-operator-controller-manager-58fc56757c-lrr9f\" (UID: \"81921157-9b41-4137-9f42-5c4e34307143\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:26.071656 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:26.071503    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81921157-9b41-4137-9f42-5c4e34307143-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-lrr9f\" (UID: \"81921157-9b41-4137-9f42-5c4e34307143\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:26.071910 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:26.071890    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81921157-9b41-4137-9f42-5c4e34307143-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-lrr9f\" (UID: \"81921157-9b41-4137-9f42-5c4e34307143\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:26.083690 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:26.083667    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92s8g\" (UniqueName: \"kubernetes.io/projected/81921157-9b41-4137-9f42-5c4e34307143-kube-api-access-92s8g\") pod \"kuadrant-operator-controller-manager-58fc56757c-lrr9f\" (UID: \"81921157-9b41-4137-9f42-5c4e34307143\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:26.242317 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:26.242233    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:26.265944 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:26.265914    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:26.527085 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:26.526994    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:26.547287 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:26.547258    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"]
Sep 06 05:05:26.550638 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:05:26.550611    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81921157_9b41_4137_9f42_5c4e34307143.slice/crio-551d498c24ef04b3b88375f347bbd440594bfac79f86462b166398c9cb63e9d8 WatchSource:0}: Error finding container 551d498c24ef04b3b88375f347bbd440594bfac79f86462b166398c9cb63e9d8: Status 404 returned error can't find the container with id 551d498c24ef04b3b88375f347bbd440594bfac79f86462b166398c9cb63e9d8
Sep 06 05:05:26.581168 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:26.581137    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f" event={"ID":"81921157-9b41-4137-9f42-5c4e34307143","Type":"ContainerStarted","Data":"551d498c24ef04b3b88375f347bbd440594bfac79f86462b166398c9cb63e9d8"}
Sep 06 05:05:26.633258 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:26.633232    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:26.719208 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:26.719169    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:28.128395 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:28.128352    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:28.383604 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:28.383519    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-slkcz"]
Sep 06 05:05:28.390281 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:28.390255    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-slkcz"
Sep 06 05:05:28.392786 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:28.392714    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-z5nn7\""
Sep 06 05:05:28.402616 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:28.402546    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-slkcz"]
Sep 06 05:05:28.491946 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:28.491907    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvxbw\" (UniqueName: \"kubernetes.io/projected/4aa5fc66-cede-4224-bb27-d4e9f85000fe-kube-api-access-gvxbw\") pod \"authorino-operator-68d88948d8-slkcz\" (UID: \"4aa5fc66-cede-4224-bb27-d4e9f85000fe\") " pod="kuadrant-system/authorino-operator-68d88948d8-slkcz"
Sep 06 05:05:28.595436 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:28.595397    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvxbw\" (UniqueName: \"kubernetes.io/projected/4aa5fc66-cede-4224-bb27-d4e9f85000fe-kube-api-access-gvxbw\") pod \"authorino-operator-68d88948d8-slkcz\" (UID: \"4aa5fc66-cede-4224-bb27-d4e9f85000fe\") " pod="kuadrant-system/authorino-operator-68d88948d8-slkcz"
Sep 06 05:05:28.604693 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:28.604636    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvxbw\" (UniqueName: \"kubernetes.io/projected/4aa5fc66-cede-4224-bb27-d4e9f85000fe-kube-api-access-gvxbw\") pod \"authorino-operator-68d88948d8-slkcz\" (UID: \"4aa5fc66-cede-4224-bb27-d4e9f85000fe\") " pod="kuadrant-system/authorino-operator-68d88948d8-slkcz"
Sep 06 05:05:28.705717 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:28.705629    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-slkcz"
Sep 06 05:05:29.251794 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:29.251758    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:29.277210 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:29.277165    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-slkcz"]
Sep 06 05:05:29.279745 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:05:29.279719    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa5fc66_cede_4224_bb27_d4e9f85000fe.slice/crio-a12e5859239ca291950aab1336f1a74c40f8f45f7b8c2eba71f6b3a1a2fb32c1 WatchSource:0}: Error finding container a12e5859239ca291950aab1336f1a74c40f8f45f7b8c2eba71f6b3a1a2fb32c1: Status 404 returned error can't find the container with id a12e5859239ca291950aab1336f1a74c40f8f45f7b8c2eba71f6b3a1a2fb32c1
Sep 06 05:05:29.471009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:29.470971    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:29.603187 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:29.603053    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-slkcz" event={"ID":"4aa5fc66-cede-4224-bb27-d4e9f85000fe","Type":"ContainerStarted","Data":"a12e5859239ca291950aab1336f1a74c40f8f45f7b8c2eba71f6b3a1a2fb32c1"}
Sep 06 05:05:30.353720 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:30.353681    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:30.456531 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:30.456490    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:30.798890 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:30.798859    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-ql9kj"]
Sep 06 05:05:30.802601 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:30.802577    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ql9kj"
Sep 06 05:05:30.807277 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:30.807255    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-klcj6\""
Sep 06 05:05:30.807686 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:30.807663    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 06 05:05:30.812413 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:30.812388    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-ql9kj"]
Sep 06 05:05:30.817207 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:30.817184    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rksh4\" (UniqueName: \"kubernetes.io/projected/b8c762b2-56e7-4071-a4d4-03d7651ee8ed-kube-api-access-rksh4\") pod \"dns-operator-controller-manager-5985564776-ql9kj\" (UID: \"b8c762b2-56e7-4071-a4d4-03d7651ee8ed\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-ql9kj"
Sep 06 05:05:30.918365 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:30.918277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rksh4\" (UniqueName: \"kubernetes.io/projected/b8c762b2-56e7-4071-a4d4-03d7651ee8ed-kube-api-access-rksh4\") pod \"dns-operator-controller-manager-5985564776-ql9kj\" (UID: \"b8c762b2-56e7-4071-a4d4-03d7651ee8ed\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-ql9kj"
Sep 06 05:05:30.926550 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:30.926521    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rksh4\" (UniqueName: \"kubernetes.io/projected/b8c762b2-56e7-4071-a4d4-03d7651ee8ed-kube-api-access-rksh4\") pod \"dns-operator-controller-manager-5985564776-ql9kj\" (UID: \"b8c762b2-56e7-4071-a4d4-03d7651ee8ed\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-ql9kj"
Sep 06 05:05:31.114490 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:31.114444    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ql9kj"
Sep 06 05:05:31.440403 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:31.440374    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:31.598084 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:31.597876    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-ql9kj"]
Sep 06 05:05:31.601166 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:05:31.601134    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c762b2_56e7_4071_a4d4_03d7651ee8ed.slice/crio-abb4fbbc5613fdb891c51f2b7db6ac84748347a0f39307f9b9a1b6ea2db719f2 WatchSource:0}: Error finding container abb4fbbc5613fdb891c51f2b7db6ac84748347a0f39307f9b9a1b6ea2db719f2: Status 404 returned error can't find the container with id abb4fbbc5613fdb891c51f2b7db6ac84748347a0f39307f9b9a1b6ea2db719f2
Sep 06 05:05:31.615301 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:31.615274    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ql9kj" event={"ID":"b8c762b2-56e7-4071-a4d4-03d7651ee8ed","Type":"ContainerStarted","Data":"abb4fbbc5613fdb891c51f2b7db6ac84748347a0f39307f9b9a1b6ea2db719f2"}
Sep 06 05:05:31.617299 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:31.617274    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-slkcz" event={"ID":"4aa5fc66-cede-4224-bb27-d4e9f85000fe","Type":"ContainerStarted","Data":"3e8d35135c9d5b128ebc712197eb121dcdda4501f68a82fc9a5f04f500a3e7be"}
Sep 06 05:05:31.617432 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:31.617339    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-slkcz"
Sep 06 05:05:31.619197 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:31.619177    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f" event={"ID":"81921157-9b41-4137-9f42-5c4e34307143","Type":"ContainerStarted","Data":"da091c901827947cc1378f6c66a3867f751f303aa5ac7a67331b053be5e1d5f2"}
Sep 06 05:05:31.619337 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:31.619325    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:31.635887 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:31.635844    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-slkcz" podStartSLOduration=1.524589399 podStartE2EDuration="3.63583055s" totalImagesPullingTime="2.111241151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:05:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:05:31.63456215 +0000 UTC m=+599.004017395" watchObservedRunningTime="2026-09-06 05:05:31.63583055 +0000 UTC m=+599.005285795"
Sep 06 05:05:31.654356 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:31.654303    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f" podStartSLOduration=2.540407586 podStartE2EDuration="6.654287755s" totalImagesPullingTime="4.113880169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:05:31.652205027 +0000 UTC m=+599.021660276" watchObservedRunningTime="2026-09-06 05:05:31.654287755 +0000 UTC m=+599.023743000"
Sep 06 05:05:32.417204 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:32.417147    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:32.512276 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:32.512238    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:32.912593 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:32.912553    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"]
Sep 06 05:05:32.916629 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:32.916607    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"
Sep 06 05:05:32.918941 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:32.918915    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-f7554\""
Sep 06 05:05:32.923326 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:32.923302    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"]
Sep 06 05:05:32.937519 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:32.937475    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rcr5\" (UniqueName: \"kubernetes.io/projected/3e76dd39-9818-40ff-8119-c253e0b98a75-kube-api-access-2rcr5\") pod \"limitador-operator-controller-manager-6c549b6d85-b4qlt\" (UID: \"3e76dd39-9818-40ff-8119-c253e0b98a75\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"
Sep 06 05:05:33.038272 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:33.038237    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rcr5\" (UniqueName: \"kubernetes.io/projected/3e76dd39-9818-40ff-8119-c253e0b98a75-kube-api-access-2rcr5\") pod \"limitador-operator-controller-manager-6c549b6d85-b4qlt\" (UID: \"3e76dd39-9818-40ff-8119-c253e0b98a75\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"
Sep 06 05:05:33.049293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:33.049261    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rcr5\" (UniqueName: \"kubernetes.io/projected/3e76dd39-9818-40ff-8119-c253e0b98a75-kube-api-access-2rcr5\") pod \"limitador-operator-controller-manager-6c549b6d85-b4qlt\" (UID: \"3e76dd39-9818-40ff-8119-c253e0b98a75\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"
Sep 06 05:05:33.183872 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:33.183847    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 06 05:05:33.232462 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:33.232438    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-f7554\""
Sep 06 05:05:33.239919 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:33.239900    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"
Sep 06 05:05:33.287230 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:33.287148    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:33.510918 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:33.510894    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"]
Sep 06 05:05:33.513210 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:05:33.513180    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e76dd39_9818_40ff_8119_c253e0b98a75.slice/crio-a4223fa141b47b0248b1fe526170558b2279f432b02f55b69047272a3727a58c WatchSource:0}: Error finding container a4223fa141b47b0248b1fe526170558b2279f432b02f55b69047272a3727a58c: Status 404 returned error can't find the container with id a4223fa141b47b0248b1fe526170558b2279f432b02f55b69047272a3727a58c
Sep 06 05:05:33.631241 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:33.631211    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ql9kj" event={"ID":"b8c762b2-56e7-4071-a4d4-03d7651ee8ed","Type":"ContainerStarted","Data":"2443bda007721899881c7d7baa592e098e7352f28e1f9729d19a215c7c31f59b"}
Sep 06 05:05:33.631435 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:33.631336    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ql9kj"
Sep 06 05:05:33.632609 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:33.632585    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt" event={"ID":"3e76dd39-9818-40ff-8119-c253e0b98a75","Type":"ContainerStarted","Data":"a4223fa141b47b0248b1fe526170558b2279f432b02f55b69047272a3727a58c"}
Sep 06 05:05:33.660188 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:33.660148    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ql9kj" podStartSLOduration=2.082146656 podStartE2EDuration="3.660136738s" totalImagesPullingTime="1.577990082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:05:33.65846945 +0000 UTC m=+601.027924695" watchObservedRunningTime="2026-09-06 05:05:33.660136738 +0000 UTC m=+601.029591984"
Sep 06 05:05:33.774609 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:33.774525    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:34.252571 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:34.252536    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:35.645644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:35.645606    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt" event={"ID":"3e76dd39-9818-40ff-8119-c253e0b98a75","Type":"ContainerStarted","Data":"6f03b2ef26521d9e9a67bfd226af14078f2373a6c8f9006afefa3b4014f998c8"}
Sep 06 05:05:35.646033 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:35.645722    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"
Sep 06 05:05:35.661005 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:35.660957    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt" podStartSLOduration=2.211005588 podStartE2EDuration="3.660943388s" totalImagesPullingTime="1.4499378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:05:35.659690895 +0000 UTC m=+603.029146131" watchObservedRunningTime="2026-09-06 05:05:35.660943388 +0000 UTC m=+603.030398634"
Sep 06 05:05:36.640262 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:36.640226    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:36.740767 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:36.740736    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:37.036207 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:37.036131    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:37.136870 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:37.136834    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:37.840756 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:37.840726    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:37.942565 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:37.942532    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:39.915084 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:39.914972    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:39.967235 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:39.967195    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:42.627583 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:42.627550    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:42.628000 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:42.627673    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-slkcz"
Sep 06 05:05:44.642334 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.642304    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ql9kj"
Sep 06 05:05:44.777056 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.777001    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"]
Sep 06 05:05:44.780394 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.780368    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"
Sep 06 05:05:44.793826 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.793798    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"]
Sep 06 05:05:44.829070 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.829031    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"]
Sep 06 05:05:44.829259 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.829235    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f" podUID="81921157-9b41-4137-9f42-5c4e34307143" containerName="manager" containerID="cri-o://da091c901827947cc1378f6c66a3867f751f303aa5ac7a67331b053be5e1d5f2" gracePeriod=2
Sep 06 05:05:44.841647 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.841617    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"]
Sep 06 05:05:44.841969 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.841948    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jmtv\" (UniqueName: \"kubernetes.io/projected/76f0adab-dc1d-40e6-935d-c5f2f034ed26-kube-api-access-9jmtv\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-78tvz\" (UID: \"76f0adab-dc1d-40e6-935d-c5f2f034ed26\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"
Sep 06 05:05:44.842075 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.841986    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76f0adab-dc1d-40e6-935d-c5f2f034ed26-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-78tvz\" (UID: \"76f0adab-dc1d-40e6-935d-c5f2f034ed26\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"
Sep 06 05:05:44.850006 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:44.849968    2578 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"81921157-9b41-4137-9f42-5c4e34307143\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-lrr9f\": pods \"kuadrant-operator-controller-manager-58fc56757c-lrr9f\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:44.852601 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.852571    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"]
Sep 06 05:05:44.853007 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.852990    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81921157-9b41-4137-9f42-5c4e34307143" containerName="manager"
Sep 06 05:05:44.853088 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.853052    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="81921157-9b41-4137-9f42-5c4e34307143" containerName="manager"
Sep 06 05:05:44.853134 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.853125    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="81921157-9b41-4137-9f42-5c4e34307143" containerName="manager"
Sep 06 05:05:44.856256 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.856240    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"
Sep 06 05:05:44.858361 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.858333    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"]
Sep 06 05:05:44.858601 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:44.858582    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-9jmtv], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz" podUID="76f0adab-dc1d-40e6-935d-c5f2f034ed26"
Sep 06 05:05:44.862708 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.862672    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"]
Sep 06 05:05:44.874408 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.874383    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"]
Sep 06 05:05:44.876644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.876623    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"]
Sep 06 05:05:44.876831 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.876797    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt" podUID="3e76dd39-9818-40ff-8119-c253e0b98a75" containerName="manager" containerID="cri-o://6f03b2ef26521d9e9a67bfd226af14078f2373a6c8f9006afefa3b4014f998c8" gracePeriod=2
Sep 06 05:05:44.878471 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.878452    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"
Sep 06 05:05:44.883556 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.883529    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"]
Sep 06 05:05:44.888218 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.888197    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"]
Sep 06 05:05:44.888352 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.888289    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"
Sep 06 05:05:44.891733 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:44.891704    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-lrr9f\" is forbidden: User \"system:node:ip-10-0-134-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-29.ec2.internal' and this object" podUID="81921157-9b41-4137-9f42-5c4e34307143" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:44.893490 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:44.893409    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-lrr9f\" is forbidden: User \"system:node:ip-10-0-134-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-29.ec2.internal' and this object" podUID="81921157-9b41-4137-9f42-5c4e34307143" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:44.901974 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.901950    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"]
Sep 06 05:05:44.906258 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.906235    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-g8q6h"]
Sep 06 05:05:44.906745 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.906727    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e76dd39-9818-40ff-8119-c253e0b98a75" containerName="manager"
Sep 06 05:05:44.906789 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.906750    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e76dd39-9818-40ff-8119-c253e0b98a75" containerName="manager"
Sep 06 05:05:44.906881 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.906869    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e76dd39-9818-40ff-8119-c253e0b98a75" containerName="manager"
Sep 06 05:05:44.910009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.909989    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-g8q6h"
Sep 06 05:05:44.918604 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.918582    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-g8q6h"]
Sep 06 05:05:44.924635 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:44.924607    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-b4qlt\" is forbidden: User \"system:node:ip-10-0-134-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-29.ec2.internal' and this object" podUID="3e76dd39-9818-40ff-8119-c253e0b98a75" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"
Sep 06 05:05:44.926612 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:44.926585    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-b4qlt\" is forbidden: User \"system:node:ip-10-0-134-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-29.ec2.internal' and this object" podUID="3e76dd39-9818-40ff-8119-c253e0b98a75" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"
Sep 06 05:05:44.928176 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:44.928155    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-lrr9f\" is forbidden: User \"system:node:ip-10-0-134-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-29.ec2.internal' and this object" podUID="81921157-9b41-4137-9f42-5c4e34307143" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:44.943381 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.943353    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jmtv\" (UniqueName: \"kubernetes.io/projected/76f0adab-dc1d-40e6-935d-c5f2f034ed26-kube-api-access-9jmtv\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-78tvz\" (UID: \"76f0adab-dc1d-40e6-935d-c5f2f034ed26\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"
Sep 06 05:05:44.943518 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.943391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnzq4\" (UniqueName: \"kubernetes.io/projected/f00abe5d-fd79-47be-965f-6b3222e576e8-kube-api-access-dnzq4\") pod \"kuadrant-operator-controller-manager-58fc56757c-c6gcs\" (UID: \"f00abe5d-fd79-47be-965f-6b3222e576e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"
Sep 06 05:05:44.943518 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.943412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76f0adab-dc1d-40e6-935d-c5f2f034ed26-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-78tvz\" (UID: \"76f0adab-dc1d-40e6-935d-c5f2f034ed26\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"
Sep 06 05:05:44.943633 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.943509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f00abe5d-fd79-47be-965f-6b3222e576e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-c6gcs\" (UID: \"f00abe5d-fd79-47be-965f-6b3222e576e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"
Sep 06 05:05:44.943633 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.943537    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1aaff57-1941-48e7-b0e1-435701602c80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-bls2h\" (UID: \"b1aaff57-1941-48e7-b0e1-435701602c80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"
Sep 06 05:05:44.943633 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.943576    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krwb2\" (UniqueName: \"kubernetes.io/projected/b1aaff57-1941-48e7-b0e1-435701602c80-kube-api-access-krwb2\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-bls2h\" (UID: \"b1aaff57-1941-48e7-b0e1-435701602c80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"
Sep 06 05:05:44.943808 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:44.943791    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76f0adab-dc1d-40e6-935d-c5f2f034ed26-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-78tvz\" (UID: \"76f0adab-dc1d-40e6-935d-c5f2f034ed26\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"
Sep 06 05:05:44.948146 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:44.948126    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9jmtv for pod kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz: failed to fetch token: pod "kuadrant-operator-controller-manager-5b65f94d4c-78tvz" not found
Sep 06 05:05:44.948236 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:44.948181    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76f0adab-dc1d-40e6-935d-c5f2f034ed26-kube-api-access-9jmtv podName:76f0adab-dc1d-40e6-935d-c5f2f034ed26 nodeName:}" failed. No retries permitted until 2026-09-06 05:05:45.448166606 +0000 UTC m=+612.817621830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9jmtv" (UniqueName: "kubernetes.io/projected/76f0adab-dc1d-40e6-935d-c5f2f034ed26-kube-api-access-9jmtv") pod "kuadrant-operator-controller-manager-5b65f94d4c-78tvz" (UID: "76f0adab-dc1d-40e6-935d-c5f2f034ed26") : failed to fetch token: pod "kuadrant-operator-controller-manager-5b65f94d4c-78tvz" not found
Sep 06 05:05:45.044648 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.044523    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f00abe5d-fd79-47be-965f-6b3222e576e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-c6gcs\" (UID: \"f00abe5d-fd79-47be-965f-6b3222e576e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"
Sep 06 05:05:45.044648 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.044571    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1aaff57-1941-48e7-b0e1-435701602c80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-bls2h\" (UID: \"b1aaff57-1941-48e7-b0e1-435701602c80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"
Sep 06 05:05:45.044648 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.044648    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krwb2\" (UniqueName: \"kubernetes.io/projected/b1aaff57-1941-48e7-b0e1-435701602c80-kube-api-access-krwb2\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-bls2h\" (UID: \"b1aaff57-1941-48e7-b0e1-435701602c80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"
Sep 06 05:05:45.044906 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.044702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fgb2\" (UniqueName: \"kubernetes.io/projected/10dbeb7b-8863-4952-be43-f23bb8533bab-kube-api-access-8fgb2\") pod \"limitador-operator-controller-manager-6c549b6d85-g8q6h\" (UID: \"10dbeb7b-8863-4952-be43-f23bb8533bab\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-g8q6h"
Sep 06 05:05:45.044906 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.044783    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnzq4\" (UniqueName: \"kubernetes.io/projected/f00abe5d-fd79-47be-965f-6b3222e576e8-kube-api-access-dnzq4\") pod \"kuadrant-operator-controller-manager-58fc56757c-c6gcs\" (UID: \"f00abe5d-fd79-47be-965f-6b3222e576e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"
Sep 06 05:05:45.045040 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.044937    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f00abe5d-fd79-47be-965f-6b3222e576e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-c6gcs\" (UID: \"f00abe5d-fd79-47be-965f-6b3222e576e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"
Sep 06 05:05:45.045040 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.044977    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1aaff57-1941-48e7-b0e1-435701602c80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-bls2h\" (UID: \"b1aaff57-1941-48e7-b0e1-435701602c80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"
Sep 06 05:05:45.057668 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.057623    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnzq4\" (UniqueName: \"kubernetes.io/projected/f00abe5d-fd79-47be-965f-6b3222e576e8-kube-api-access-dnzq4\") pod \"kuadrant-operator-controller-manager-58fc56757c-c6gcs\" (UID: \"f00abe5d-fd79-47be-965f-6b3222e576e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"
Sep 06 05:05:45.057811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.057784    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krwb2\" (UniqueName: \"kubernetes.io/projected/b1aaff57-1941-48e7-b0e1-435701602c80-kube-api-access-krwb2\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-bls2h\" (UID: \"b1aaff57-1941-48e7-b0e1-435701602c80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"
Sep 06 05:05:45.107407 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.107379    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:45.109388 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.109371    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"
Sep 06 05:05:45.109594 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:45.109578    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-lrr9f\" is forbidden: User \"system:node:ip-10-0-134-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-29.ec2.internal' and this object" podUID="81921157-9b41-4137-9f42-5c4e34307143" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:45.111377 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:45.111356    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-b4qlt\" is forbidden: User \"system:node:ip-10-0-134-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-29.ec2.internal' and this object" podUID="3e76dd39-9818-40ff-8119-c253e0b98a75" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"
Sep 06 05:05:45.113126 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:45.113106    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-b4qlt\" is forbidden: User \"system:node:ip-10-0-134-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-29.ec2.internal' and this object" podUID="3e76dd39-9818-40ff-8119-c253e0b98a75" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"
Sep 06 05:05:45.114941 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:45.114920    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-lrr9f\" is forbidden: User \"system:node:ip-10-0-134-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-29.ec2.internal' and this object" podUID="81921157-9b41-4137-9f42-5c4e34307143" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:45.145717 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.145640    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fgb2\" (UniqueName: \"kubernetes.io/projected/10dbeb7b-8863-4952-be43-f23bb8533bab-kube-api-access-8fgb2\") pod \"limitador-operator-controller-manager-6c549b6d85-g8q6h\" (UID: \"10dbeb7b-8863-4952-be43-f23bb8533bab\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-g8q6h"
Sep 06 05:05:45.155333 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.155300    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fgb2\" (UniqueName: \"kubernetes.io/projected/10dbeb7b-8863-4952-be43-f23bb8533bab-kube-api-access-8fgb2\") pod \"limitador-operator-controller-manager-6c549b6d85-g8q6h\" (UID: \"10dbeb7b-8863-4952-be43-f23bb8533bab\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-g8q6h"
Sep 06 05:05:45.246152 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.246119    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rcr5\" (UniqueName: \"kubernetes.io/projected/3e76dd39-9818-40ff-8119-c253e0b98a75-kube-api-access-2rcr5\") pod \"3e76dd39-9818-40ff-8119-c253e0b98a75\" (UID: \"3e76dd39-9818-40ff-8119-c253e0b98a75\") "
Sep 06 05:05:45.246152 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.246154    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92s8g\" (UniqueName: \"kubernetes.io/projected/81921157-9b41-4137-9f42-5c4e34307143-kube-api-access-92s8g\") pod \"81921157-9b41-4137-9f42-5c4e34307143\" (UID: \"81921157-9b41-4137-9f42-5c4e34307143\") "
Sep 06 05:05:45.246392 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.246178    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81921157-9b41-4137-9f42-5c4e34307143-extensions-socket-volume\") pod \"81921157-9b41-4137-9f42-5c4e34307143\" (UID: \"81921157-9b41-4137-9f42-5c4e34307143\") "
Sep 06 05:05:45.246632 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.246601    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81921157-9b41-4137-9f42-5c4e34307143-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "81921157-9b41-4137-9f42-5c4e34307143" (UID: "81921157-9b41-4137-9f42-5c4e34307143"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:05:45.248413 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.248391    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81921157-9b41-4137-9f42-5c4e34307143-kube-api-access-92s8g" (OuterVolumeSpecName: "kube-api-access-92s8g") pod "81921157-9b41-4137-9f42-5c4e34307143" (UID: "81921157-9b41-4137-9f42-5c4e34307143"). InnerVolumeSpecName "kube-api-access-92s8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:05:45.248485 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.248441    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e76dd39-9818-40ff-8119-c253e0b98a75-kube-api-access-2rcr5" (OuterVolumeSpecName: "kube-api-access-2rcr5") pod "3e76dd39-9818-40ff-8119-c253e0b98a75" (UID: "3e76dd39-9818-40ff-8119-c253e0b98a75"). InnerVolumeSpecName "kube-api-access-2rcr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:05:45.269588 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.269563    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"
Sep 06 05:05:45.275762 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.275742    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"
Sep 06 05:05:45.281001 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.280982    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-g8q6h"
Sep 06 05:05:45.347531 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.347506    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rcr5\" (UniqueName: \"kubernetes.io/projected/3e76dd39-9818-40ff-8119-c253e0b98a75-kube-api-access-2rcr5\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:05:45.347531 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.347531    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92s8g\" (UniqueName: \"kubernetes.io/projected/81921157-9b41-4137-9f42-5c4e34307143-kube-api-access-92s8g\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:05:45.347666 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.347541    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81921157-9b41-4137-9f42-5c4e34307143-extensions-socket-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:05:45.549423 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.549346    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jmtv\" (UniqueName: \"kubernetes.io/projected/76f0adab-dc1d-40e6-935d-c5f2f034ed26-kube-api-access-9jmtv\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-78tvz\" (UID: \"76f0adab-dc1d-40e6-935d-c5f2f034ed26\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"
Sep 06 05:05:45.551814 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:45.551781    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9jmtv for pod kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz: failed to fetch token: pod "kuadrant-operator-controller-manager-5b65f94d4c-78tvz" not found
Sep 06 05:05:45.551942 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:45.551866    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76f0adab-dc1d-40e6-935d-c5f2f034ed26-kube-api-access-9jmtv podName:76f0adab-dc1d-40e6-935d-c5f2f034ed26 nodeName:}" failed. No retries permitted until 2026-09-06 05:05:46.551846005 +0000 UTC m=+613.921301228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9jmtv" (UniqueName: "kubernetes.io/projected/76f0adab-dc1d-40e6-935d-c5f2f034ed26-kube-api-access-9jmtv") pod "kuadrant-operator-controller-manager-5b65f94d4c-78tvz" (UID: "76f0adab-dc1d-40e6-935d-c5f2f034ed26") : failed to fetch token: pod "kuadrant-operator-controller-manager-5b65f94d4c-78tvz" not found
Sep 06 05:05:45.553921 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.553900    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"]
Sep 06 05:05:45.556286 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:05:45.556262    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf00abe5d_fd79_47be_965f_6b3222e576e8.slice/crio-2740dcb69adf1b1491ddf35acaa7d6d90f12925743238b083b2c67ecd744c947 WatchSource:0}: Error finding container 2740dcb69adf1b1491ddf35acaa7d6d90f12925743238b083b2c67ecd744c947: Status 404 returned error can't find the container with id 2740dcb69adf1b1491ddf35acaa7d6d90f12925743238b083b2c67ecd744c947
Sep 06 05:05:45.560521 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.560496    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-g8q6h"]
Sep 06 05:05:45.562512 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:05:45.562486    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10dbeb7b_8863_4952_be43_f23bb8533bab.slice/crio-e1288ec8540977190e5a86fdf80c9a6f73bae2b73ac6469754237f700a91558a WatchSource:0}: Error finding container e1288ec8540977190e5a86fdf80c9a6f73bae2b73ac6469754237f700a91558a: Status 404 returned error can't find the container with id e1288ec8540977190e5a86fdf80c9a6f73bae2b73ac6469754237f700a91558a
Sep 06 05:05:45.562775 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.562758    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"]
Sep 06 05:05:45.565974 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:05:45.565956    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1aaff57_1941_48e7_b0e1_435701602c80.slice/crio-b3dcefe3c563a32131547c448048a1a7d687f5f8e8c844a19ee391a866006782 WatchSource:0}: Error finding container b3dcefe3c563a32131547c448048a1a7d687f5f8e8c844a19ee391a866006782: Status 404 returned error can't find the container with id b3dcefe3c563a32131547c448048a1a7d687f5f8e8c844a19ee391a866006782
Sep 06 05:05:45.699801 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.699567    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-g8q6h" event={"ID":"10dbeb7b-8863-4952-be43-f23bb8533bab","Type":"ContainerStarted","Data":"bee41e1d3d7a8dc63aa8744bfae0be176689c6f86d2d5e726c3e8c4dd83c5c41"}
Sep 06 05:05:45.699801 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.699634    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-g8q6h" event={"ID":"10dbeb7b-8863-4952-be43-f23bb8533bab","Type":"ContainerStarted","Data":"e1288ec8540977190e5a86fdf80c9a6f73bae2b73ac6469754237f700a91558a"}
Sep 06 05:05:45.699801 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.699700    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-g8q6h"
Sep 06 05:05:45.701999 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.701939    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs" event={"ID":"f00abe5d-fd79-47be-965f-6b3222e576e8","Type":"ContainerStarted","Data":"949fd59cdfa70b4fd413ed5533a5abdeca467572e2599c62fc818b9deaa7159b"}
Sep 06 05:05:45.702160 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.702051    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs" event={"ID":"f00abe5d-fd79-47be-965f-6b3222e576e8","Type":"ContainerStarted","Data":"2740dcb69adf1b1491ddf35acaa7d6d90f12925743238b083b2c67ecd744c947"}
Sep 06 05:05:45.702160 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.702086    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"
Sep 06 05:05:45.703630 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.703606    2578 generic.go:356] "Generic (PLEG): container finished" podID="81921157-9b41-4137-9f42-5c4e34307143" containerID="da091c901827947cc1378f6c66a3867f751f303aa5ac7a67331b053be5e1d5f2" exitCode=0
Sep 06 05:05:45.703745 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.703646    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lrr9f"
Sep 06 05:05:45.703745 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.703660    2578 scope.go:117] "RemoveContainer" containerID="da091c901827947cc1378f6c66a3867f751f303aa5ac7a67331b053be5e1d5f2"
Sep 06 05:05:45.705497 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.705464    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h" event={"ID":"b1aaff57-1941-48e7-b0e1-435701602c80","Type":"ContainerStarted","Data":"15582671f95fdff0d75a4bea3d735a9f76f23dc1314ec3deb718ca3befbced1a"}
Sep 06 05:05:45.705626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.705529    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"
Sep 06 05:05:45.705626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.705554    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h" event={"ID":"b1aaff57-1941-48e7-b0e1-435701602c80","Type":"ContainerStarted","Data":"b3dcefe3c563a32131547c448048a1a7d687f5f8e8c844a19ee391a866006782"}
Sep 06 05:05:45.707145 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.707123    2578 generic.go:356] "Generic (PLEG): container finished" podID="3e76dd39-9818-40ff-8119-c253e0b98a75" containerID="6f03b2ef26521d9e9a67bfd226af14078f2373a6c8f9006afefa3b4014f998c8" exitCode=0
Sep 06 05:05:45.707231 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.707162    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-b4qlt"
Sep 06 05:05:45.707231 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.707175    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"
Sep 06 05:05:45.714217 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.714198    2578 scope.go:117] "RemoveContainer" containerID="da091c901827947cc1378f6c66a3867f751f303aa5ac7a67331b053be5e1d5f2"
Sep 06 05:05:45.714547 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:45.714526    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da091c901827947cc1378f6c66a3867f751f303aa5ac7a67331b053be5e1d5f2\": container with ID starting with da091c901827947cc1378f6c66a3867f751f303aa5ac7a67331b053be5e1d5f2 not found: ID does not exist" containerID="da091c901827947cc1378f6c66a3867f751f303aa5ac7a67331b053be5e1d5f2"
Sep 06 05:05:45.714659 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.714556    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da091c901827947cc1378f6c66a3867f751f303aa5ac7a67331b053be5e1d5f2"} err="failed to get container status \"da091c901827947cc1378f6c66a3867f751f303aa5ac7a67331b053be5e1d5f2\": rpc error: code = NotFound desc = could not find container \"da091c901827947cc1378f6c66a3867f751f303aa5ac7a67331b053be5e1d5f2\": container with ID starting with da091c901827947cc1378f6c66a3867f751f303aa5ac7a67331b053be5e1d5f2 not found: ID does not exist"
Sep 06 05:05:45.714659 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.714575    2578 scope.go:117] "RemoveContainer" containerID="6f03b2ef26521d9e9a67bfd226af14078f2373a6c8f9006afefa3b4014f998c8"
Sep 06 05:05:45.717401 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.717382    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"
Sep 06 05:05:45.721266 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.721228    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-g8q6h" podStartSLOduration=1.7212153369999998 podStartE2EDuration="1.721215337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:05:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:05:45.719953528 +0000 UTC m=+613.089408775" watchObservedRunningTime="2026-09-06 05:05:45.721215337 +0000 UTC m=+613.090670580"
Sep 06 05:05:45.721870 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:45.721841    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5b65f94d4c-78tvz\" is forbidden: User \"system:node:ip-10-0-134-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-29.ec2.internal' and this object" podUID="76f0adab-dc1d-40e6-935d-c5f2f034ed26" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"
Sep 06 05:05:45.722997 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.722980    2578 scope.go:117] "RemoveContainer" containerID="6f03b2ef26521d9e9a67bfd226af14078f2373a6c8f9006afefa3b4014f998c8"
Sep 06 05:05:45.723291 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:45.723274    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f03b2ef26521d9e9a67bfd226af14078f2373a6c8f9006afefa3b4014f998c8\": container with ID starting with 6f03b2ef26521d9e9a67bfd226af14078f2373a6c8f9006afefa3b4014f998c8 not found: ID does not exist" containerID="6f03b2ef26521d9e9a67bfd226af14078f2373a6c8f9006afefa3b4014f998c8"
Sep 06 05:05:45.723339 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.723299    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f03b2ef26521d9e9a67bfd226af14078f2373a6c8f9006afefa3b4014f998c8"} err="failed to get container status \"6f03b2ef26521d9e9a67bfd226af14078f2373a6c8f9006afefa3b4014f998c8\": rpc error: code = NotFound desc = could not find container \"6f03b2ef26521d9e9a67bfd226af14078f2373a6c8f9006afefa3b4014f998c8\": container with ID starting with 6f03b2ef26521d9e9a67bfd226af14078f2373a6c8f9006afefa3b4014f998c8 not found: ID does not exist"
Sep 06 05:05:45.743713 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.743665    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h" podStartSLOduration=1.743652365 podStartE2EDuration="1.743652365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:05:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:05:45.742849647 +0000 UTC m=+613.112304893" watchObservedRunningTime="2026-09-06 05:05:45.743652365 +0000 UTC m=+613.113107602"
Sep 06 05:05:45.764065 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.763422    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs" podStartSLOduration=1.7634067930000001 podStartE2EDuration="1.763406793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:05:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:05:45.762751104 +0000 UTC m=+613.132206352" watchObservedRunningTime="2026-09-06 05:05:45.763406793 +0000 UTC m=+613.132862041"
Sep 06 05:05:45.852824 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.852787    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76f0adab-dc1d-40e6-935d-c5f2f034ed26-extensions-socket-volume\") pod \"76f0adab-dc1d-40e6-935d-c5f2f034ed26\" (UID: \"76f0adab-dc1d-40e6-935d-c5f2f034ed26\") "
Sep 06 05:05:45.853120 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.853080    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76f0adab-dc1d-40e6-935d-c5f2f034ed26-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "76f0adab-dc1d-40e6-935d-c5f2f034ed26" (UID: "76f0adab-dc1d-40e6-935d-c5f2f034ed26"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:05:45.853234 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.853098    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jmtv\" (UniqueName: \"kubernetes.io/projected/76f0adab-dc1d-40e6-935d-c5f2f034ed26-kube-api-access-9jmtv\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:05:45.952850 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.952806    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:45.953850 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:45.953832    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76f0adab-dc1d-40e6-935d-c5f2f034ed26-extensions-socket-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:05:46.049325 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:46.049292    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:46.714190 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:46.714162    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"
Sep 06 05:05:46.716547 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:46.716516    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5b65f94d4c-78tvz\" is forbidden: User \"system:node:ip-10-0-134-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-29.ec2.internal' and this object" podUID="76f0adab-dc1d-40e6-935d-c5f2f034ed26" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"
Sep 06 05:05:46.721697 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:46.721674    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5b65f94d4c-78tvz\" is forbidden: User \"system:node:ip-10-0-134-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-29.ec2.internal' and this object" podUID="76f0adab-dc1d-40e6-935d-c5f2f034ed26" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-78tvz"
Sep 06 05:05:47.152974 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:47.152946    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e76dd39-9818-40ff-8119-c253e0b98a75" path="/var/lib/kubelet/pods/3e76dd39-9818-40ff-8119-c253e0b98a75/volumes"
Sep 06 05:05:47.153297 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:47.153284    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76f0adab-dc1d-40e6-935d-c5f2f034ed26" path="/var/lib/kubelet/pods/76f0adab-dc1d-40e6-935d-c5f2f034ed26/volumes"
Sep 06 05:05:47.153503 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:47.153491    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81921157-9b41-4137-9f42-5c4e34307143" path="/var/lib/kubelet/pods/81921157-9b41-4137-9f42-5c4e34307143/volumes"
Sep 06 05:05:48.305476 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:48.305442    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:48.406144 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:48.406112    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:53.049296 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:53.049257    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:53.151948 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:53.151910    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:56.716400 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:56.716369    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-g8q6h"
Sep 06 05:05:56.716858 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:56.716437    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"
Sep 06 05:05:56.716858 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:56.716470    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"
Sep 06 05:05:56.802351 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:56.802310    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"]
Sep 06 05:05:56.802532 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:56.802501    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs" podUID="f00abe5d-fd79-47be-965f-6b3222e576e8" containerName="manager" containerID="cri-o://949fd59cdfa70b4fd413ed5533a5abdeca467572e2599c62fc818b9deaa7159b" gracePeriod=10
Sep 06 05:05:57.039652 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.039628    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"
Sep 06 05:05:57.153213 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.153185    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnzq4\" (UniqueName: \"kubernetes.io/projected/f00abe5d-fd79-47be-965f-6b3222e576e8-kube-api-access-dnzq4\") pod \"f00abe5d-fd79-47be-965f-6b3222e576e8\" (UID: \"f00abe5d-fd79-47be-965f-6b3222e576e8\") "
Sep 06 05:05:57.153406 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.153310    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f00abe5d-fd79-47be-965f-6b3222e576e8-extensions-socket-volume\") pod \"f00abe5d-fd79-47be-965f-6b3222e576e8\" (UID: \"f00abe5d-fd79-47be-965f-6b3222e576e8\") "
Sep 06 05:05:57.153710 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.153686    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f00abe5d-fd79-47be-965f-6b3222e576e8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f00abe5d-fd79-47be-965f-6b3222e576e8" (UID: "f00abe5d-fd79-47be-965f-6b3222e576e8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:05:57.155225 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.155200    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f00abe5d-fd79-47be-965f-6b3222e576e8-kube-api-access-dnzq4" (OuterVolumeSpecName: "kube-api-access-dnzq4") pod "f00abe5d-fd79-47be-965f-6b3222e576e8" (UID: "f00abe5d-fd79-47be-965f-6b3222e576e8"). InnerVolumeSpecName "kube-api-access-dnzq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:05:57.254601 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.254568    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f00abe5d-fd79-47be-965f-6b3222e576e8-extensions-socket-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:05:57.254601 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.254594    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnzq4\" (UniqueName: \"kubernetes.io/projected/f00abe5d-fd79-47be-965f-6b3222e576e8-kube-api-access-dnzq4\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:05:57.770762 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.770724    2578 generic.go:356] "Generic (PLEG): container finished" podID="f00abe5d-fd79-47be-965f-6b3222e576e8" containerID="949fd59cdfa70b4fd413ed5533a5abdeca467572e2599c62fc818b9deaa7159b" exitCode=0
Sep 06 05:05:57.771145 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.770783    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"
Sep 06 05:05:57.771145 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.770793    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs" event={"ID":"f00abe5d-fd79-47be-965f-6b3222e576e8","Type":"ContainerDied","Data":"949fd59cdfa70b4fd413ed5533a5abdeca467572e2599c62fc818b9deaa7159b"}
Sep 06 05:05:57.771145 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.770864    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs" event={"ID":"f00abe5d-fd79-47be-965f-6b3222e576e8","Type":"ContainerDied","Data":"2740dcb69adf1b1491ddf35acaa7d6d90f12925743238b083b2c67ecd744c947"}
Sep 06 05:05:57.771145 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.770886    2578 scope.go:117] "RemoveContainer" containerID="949fd59cdfa70b4fd413ed5533a5abdeca467572e2599c62fc818b9deaa7159b"
Sep 06 05:05:57.780181 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.780160    2578 scope.go:117] "RemoveContainer" containerID="949fd59cdfa70b4fd413ed5533a5abdeca467572e2599c62fc818b9deaa7159b"
Sep 06 05:05:57.780454 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:05:57.780434    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"949fd59cdfa70b4fd413ed5533a5abdeca467572e2599c62fc818b9deaa7159b\": container with ID starting with 949fd59cdfa70b4fd413ed5533a5abdeca467572e2599c62fc818b9deaa7159b not found: ID does not exist" containerID="949fd59cdfa70b4fd413ed5533a5abdeca467572e2599c62fc818b9deaa7159b"
Sep 06 05:05:57.780513 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.780468    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"949fd59cdfa70b4fd413ed5533a5abdeca467572e2599c62fc818b9deaa7159b"} err="failed to get container status \"949fd59cdfa70b4fd413ed5533a5abdeca467572e2599c62fc818b9deaa7159b\": rpc error: code = NotFound desc = could not find container \"949fd59cdfa70b4fd413ed5533a5abdeca467572e2599c62fc818b9deaa7159b\": container with ID starting with 949fd59cdfa70b4fd413ed5533a5abdeca467572e2599c62fc818b9deaa7159b not found: ID does not exist"
Sep 06 05:05:57.795514 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.795491    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"]
Sep 06 05:05:57.801160 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.801139    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-c6gcs"]
Sep 06 05:05:57.856336 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.856303    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:57.955376 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:57.955343    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:58.058546 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.058464    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:58.348492 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.348403    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fff8d58f8-95th4"]
Sep 06 05:05:58.348850 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.348831    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f00abe5d-fd79-47be-965f-6b3222e576e8" containerName="manager"
Sep 06 05:05:58.348931 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.348853    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f00abe5d-fd79-47be-965f-6b3222e576e8" containerName="manager"
Sep 06 05:05:58.348989 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.348957    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f00abe5d-fd79-47be-965f-6b3222e576e8" containerName="manager"
Sep 06 05:05:58.351967 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.351946    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.361040 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.360989    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fff8d58f8-95th4"]
Sep 06 05:05:58.410444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.410414    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:58.465723 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.465688    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc4ea106-4f5d-4499-a964-564510515a9f-oauth-serving-cert\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.465893 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.465729    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc4ea106-4f5d-4499-a964-564510515a9f-trusted-ca-bundle\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.465893 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.465800    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc4ea106-4f5d-4499-a964-564510515a9f-service-ca\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.465893 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.465872    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc4ea106-4f5d-4499-a964-564510515a9f-console-config\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.466088 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.465896    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n7lb\" (UniqueName: \"kubernetes.io/projected/bc4ea106-4f5d-4499-a964-564510515a9f-kube-api-access-6n7lb\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.466088 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.465951    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc4ea106-4f5d-4499-a964-564510515a9f-console-oauth-config\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.466088 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.465999    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc4ea106-4f5d-4499-a964-564510515a9f-console-serving-cert\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.512956 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.512925    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:58.567173 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.567132    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc4ea106-4f5d-4499-a964-564510515a9f-console-config\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.567173 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.567177    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n7lb\" (UniqueName: \"kubernetes.io/projected/bc4ea106-4f5d-4499-a964-564510515a9f-kube-api-access-6n7lb\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.567344 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.567198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc4ea106-4f5d-4499-a964-564510515a9f-console-oauth-config\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.567344 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.567221    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc4ea106-4f5d-4499-a964-564510515a9f-console-serving-cert\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.567344 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.567254    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc4ea106-4f5d-4499-a964-564510515a9f-oauth-serving-cert\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.567344 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.567273    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc4ea106-4f5d-4499-a964-564510515a9f-trusted-ca-bundle\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.567344 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.567308    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc4ea106-4f5d-4499-a964-564510515a9f-service-ca\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.568085 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.568060    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc4ea106-4f5d-4499-a964-564510515a9f-console-config\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.568202 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.568118    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc4ea106-4f5d-4499-a964-564510515a9f-oauth-serving-cert\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.568202 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.568156    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc4ea106-4f5d-4499-a964-564510515a9f-service-ca\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.568318 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.568262    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc4ea106-4f5d-4499-a964-564510515a9f-trusted-ca-bundle\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.569699 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.569672    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc4ea106-4f5d-4499-a964-564510515a9f-console-serving-cert\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.569699 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.569689    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc4ea106-4f5d-4499-a964-564510515a9f-console-oauth-config\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.574406 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.574385    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n7lb\" (UniqueName: \"kubernetes.io/projected/bc4ea106-4f5d-4499-a964-564510515a9f-kube-api-access-6n7lb\") pod \"console-7fff8d58f8-95th4\" (UID: \"bc4ea106-4f5d-4499-a964-564510515a9f\") " pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.611795 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.611728    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:58.660753 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.660719    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:05:58.930350 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:58.930308    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fff8d58f8-95th4"]
Sep 06 05:05:58.934264 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:05:58.934233    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc4ea106_4f5d_4499_a964_564510515a9f.slice/crio-e06cd67a5aaf3c43a410c49858a934c46ca3eb9493f969e5c066da8dd5f38615 WatchSource:0}: Error finding container e06cd67a5aaf3c43a410c49858a934c46ca3eb9493f969e5c066da8dd5f38615: Status 404 returned error can't find the container with id e06cd67a5aaf3c43a410c49858a934c46ca3eb9493f969e5c066da8dd5f38615
Sep 06 05:05:59.145254 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:59.145221    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:59.153660 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:59.153631    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f00abe5d-fd79-47be-965f-6b3222e576e8" path="/var/lib/kubelet/pods/f00abe5d-fd79-47be-965f-6b3222e576e8/volumes"
Sep 06 05:05:59.193987 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:59.193919    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:05:59.782995 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:59.782964    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fff8d58f8-95th4" event={"ID":"bc4ea106-4f5d-4499-a964-564510515a9f","Type":"ContainerStarted","Data":"9ef4ba7fee8927f9cd5d92b30ec23b163b8e0e534e4517530210755ae1545012"}
Sep 06 05:05:59.783184 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:59.783010    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fff8d58f8-95th4" event={"ID":"bc4ea106-4f5d-4499-a964-564510515a9f","Type":"ContainerStarted","Data":"e06cd67a5aaf3c43a410c49858a934c46ca3eb9493f969e5c066da8dd5f38615"}
Sep 06 05:05:59.798773 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:05:59.798725    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7fff8d58f8-95th4" podStartSLOduration=1.798710939 podStartE2EDuration="1.798710939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:05:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:05:59.797320875 +0000 UTC m=+627.166776116" watchObservedRunningTime="2026-09-06 05:05:59.798710939 +0000 UTC m=+627.168166184"
Sep 06 05:06:06.666319 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:06.666280    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:06.761254 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:06.761223    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:08.661796 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:08.661756    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:06:08.662202 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:08.661846    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:06:08.666515 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:08.666494    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:06:08.829753 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:08.829726    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7fff8d58f8-95th4"
Sep 06 05:06:08.872142 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:08.872109    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69fdc6d5bd-b4dmg"]
Sep 06 05:06:08.966211 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:08.966120    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:09.101629 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:09.101596    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:09.202513 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:09.202483    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:09.302945 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:09.302859    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:13.141416 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.141378    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"]
Sep 06 05:06:13.145940 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.145916    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.148376 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.148343    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-nn25t\""
Sep 06 05:06:13.159471 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.158869    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"]
Sep 06 05:06:13.165629 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.165606    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:13.191314 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.191281    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58864663-ba1f-4c9a-bebd-0a016c980b43-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.191444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.191327    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.191444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.191353    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5jk9\" (UniqueName: \"kubernetes.io/projected/58864663-ba1f-4c9a-bebd-0a016c980b43-kube-api-access-n5jk9\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.191444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.191374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58864663-ba1f-4c9a-bebd-0a016c980b43-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.191444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.191412    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.191444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.191434    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58864663-ba1f-4c9a-bebd-0a016c980b43-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.191604 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.191454    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.191604 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.191478    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.191604 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.191521    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.292049 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.291989    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58864663-ba1f-4c9a-bebd-0a016c980b43-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.292225 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.292103    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.292225 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.292130    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5jk9\" (UniqueName: \"kubernetes.io/projected/58864663-ba1f-4c9a-bebd-0a016c980b43-kube-api-access-n5jk9\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.292225 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.292150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58864663-ba1f-4c9a-bebd-0a016c980b43-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.292225 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.292195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.292225 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.292217    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58864663-ba1f-4c9a-bebd-0a016c980b43-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.292490 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.292245    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.292490 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.292288    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.292490 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.292328    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.292606 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.292508    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.292732 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.292708    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.292786 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.292734    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58864663-ba1f-4c9a-bebd-0a016c980b43-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.292829 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.292790    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.292937 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.292918    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.294637 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.294616    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58864663-ba1f-4c9a-bebd-0a016c980b43-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.294783 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.294763    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58864663-ba1f-4c9a-bebd-0a016c980b43-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.299629 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.299603    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58864663-ba1f-4c9a-bebd-0a016c980b43-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.299728 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.299628    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5jk9\" (UniqueName: \"kubernetes.io/projected/58864663-ba1f-4c9a-bebd-0a016c980b43-kube-api-access-n5jk9\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8dcv\" (UID: \"58864663-ba1f-4c9a-bebd-0a016c980b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.360896 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.360854    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:13.459927 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.459844    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:13.731488 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.731405    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"]
Sep 06 05:06:13.734747 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:06:13.734715    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58864663_ba1f_4c9a_bebd_0a016c980b43.slice/crio-fa60baa9263f1619ad01ec54170746223c2c1dc4f1f8f26ab77a485e199e6338 WatchSource:0}: Error finding container fa60baa9263f1619ad01ec54170746223c2c1dc4f1f8f26ab77a485e199e6338: Status 404 returned error can't find the container with id fa60baa9263f1619ad01ec54170746223c2c1dc4f1f8f26ab77a485e199e6338
Sep 06 05:06:13.736857 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.736822    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:06:13.736914 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.736894    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:06:13.736950 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.736922    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:06:13.849780 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.849747    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:13.854338 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.854285    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv" event={"ID":"58864663-ba1f-4c9a-bebd-0a016c980b43","Type":"ContainerStarted","Data":"8b23de40a40eab76da54ea2a93667aede6e94253d4421e1b03b40a0072657301"}
Sep 06 05:06:13.854485 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.854380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv" event={"ID":"58864663-ba1f-4c9a-bebd-0a016c980b43","Type":"ContainerStarted","Data":"fa60baa9263f1619ad01ec54170746223c2c1dc4f1f8f26ab77a485e199e6338"}
Sep 06 05:06:13.878238 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.878177    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv" podStartSLOduration=0.878155329 podStartE2EDuration="878.155329ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:06:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:06:13.87584123 +0000 UTC m=+641.245296476" watchObservedRunningTime="2026-09-06 05:06:13.878155329 +0000 UTC m=+641.247610578"
Sep 06 05:06:13.951954 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:13.951923    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:14.460353 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:14.460317    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:14.464888 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:14.464863    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:14.859564 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:14.859493    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:14.860413 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:14.860391    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:06:14.905185 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:14.905153    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:14.912543 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:14.912519    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"]
Sep 06 05:06:14.915927 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:14.915909    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"
Sep 06 05:06:14.918140 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:14.918119    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2tg77\""
Sep 06 05:06:14.918256 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:14.918119    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 06 05:06:14.922805 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:14.922781    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"]
Sep 06 05:06:15.005507 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.005470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8bdaa50f-943f-4b93-8fa0-3787fab7191f-config-file\") pod \"limitador-limitador-859c4d96d6-7h4z2\" (UID: \"8bdaa50f-943f-4b93-8fa0-3787fab7191f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"
Sep 06 05:06:15.005685 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.005599    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj4xd\" (UniqueName: \"kubernetes.io/projected/8bdaa50f-943f-4b93-8fa0-3787fab7191f-kube-api-access-vj4xd\") pod \"limitador-limitador-859c4d96d6-7h4z2\" (UID: \"8bdaa50f-943f-4b93-8fa0-3787fab7191f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"
Sep 06 05:06:15.016945 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.016915    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:15.021052 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.021006    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"]
Sep 06 05:06:15.106237 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.106198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8bdaa50f-943f-4b93-8fa0-3787fab7191f-config-file\") pod \"limitador-limitador-859c4d96d6-7h4z2\" (UID: \"8bdaa50f-943f-4b93-8fa0-3787fab7191f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"
Sep 06 05:06:15.106437 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.106310    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj4xd\" (UniqueName: \"kubernetes.io/projected/8bdaa50f-943f-4b93-8fa0-3787fab7191f-kube-api-access-vj4xd\") pod \"limitador-limitador-859c4d96d6-7h4z2\" (UID: \"8bdaa50f-943f-4b93-8fa0-3787fab7191f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"
Sep 06 05:06:15.107074 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.107048    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8bdaa50f-943f-4b93-8fa0-3787fab7191f-config-file\") pod \"limitador-limitador-859c4d96d6-7h4z2\" (UID: \"8bdaa50f-943f-4b93-8fa0-3787fab7191f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"
Sep 06 05:06:15.109070 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.109041    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:15.114969 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.114881    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj4xd\" (UniqueName: \"kubernetes.io/projected/8bdaa50f-943f-4b93-8fa0-3787fab7191f-kube-api-access-vj4xd\") pod \"limitador-limitador-859c4d96d6-7h4z2\" (UID: \"8bdaa50f-943f-4b93-8fa0-3787fab7191f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"
Sep 06 05:06:15.229728 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.229690    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"
Sep 06 05:06:15.501064 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.501034    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"]
Sep 06 05:06:15.503463 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:06:15.503436    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bdaa50f_943f_4b93_8fa0_3787fab7191f.slice/crio-043491c9d894de102fc48b2dc08a03ecdcd4de8435660d375f9ce7fa759fa731 WatchSource:0}: Error finding container 043491c9d894de102fc48b2dc08a03ecdcd4de8435660d375f9ce7fa759fa731: Status 404 returned error can't find the container with id 043491c9d894de102fc48b2dc08a03ecdcd4de8435660d375f9ce7fa759fa731
Sep 06 05:06:15.505858 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.505839    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 05:06:15.698080 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.698046    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-zv545"]
Sep 06 05:06:15.702483 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.702463    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-zv545"
Sep 06 05:06:15.704650 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.704511    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-phjph\""
Sep 06 05:06:15.706536 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.706510    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-zv545"]
Sep 06 05:06:15.711960 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.711939    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtln2\" (UniqueName: \"kubernetes.io/projected/615696ff-75b3-4999-900a-087a50282695-kube-api-access-jtln2\") pod \"authorino-b54df94b4-zv545\" (UID: \"615696ff-75b3-4999-900a-087a50282695\") " pod="kuadrant-system/authorino-b54df94b4-zv545"
Sep 06 05:06:15.730148 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.730122    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:15.813278 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.813198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtln2\" (UniqueName: \"kubernetes.io/projected/615696ff-75b3-4999-900a-087a50282695-kube-api-access-jtln2\") pod \"authorino-b54df94b4-zv545\" (UID: \"615696ff-75b3-4999-900a-087a50282695\") " pod="kuadrant-system/authorino-b54df94b4-zv545"
Sep 06 05:06:15.820404 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.820373    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtln2\" (UniqueName: \"kubernetes.io/projected/615696ff-75b3-4999-900a-087a50282695-kube-api-access-jtln2\") pod \"authorino-b54df94b4-zv545\" (UID: \"615696ff-75b3-4999-900a-087a50282695\") " pod="kuadrant-system/authorino-b54df94b4-zv545"
Sep 06 05:06:15.830115 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.830092    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:15.865501 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:15.865470    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2" event={"ID":"8bdaa50f-943f-4b93-8fa0-3787fab7191f","Type":"ContainerStarted","Data":"043491c9d894de102fc48b2dc08a03ecdcd4de8435660d375f9ce7fa759fa731"}
Sep 06 05:06:16.012212 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:16.012180    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-zv545"
Sep 06 05:06:16.283867 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:16.283842    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-zv545"]
Sep 06 05:06:16.286029 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:06:16.285987    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod615696ff_75b3_4999_900a_087a50282695.slice/crio-3472cc9d381404d5c241a6dccf55b954b9a3d11f678249f00c35762137df276d WatchSource:0}: Error finding container 3472cc9d381404d5c241a6dccf55b954b9a3d11f678249f00c35762137df276d: Status 404 returned error can't find the container with id 3472cc9d381404d5c241a6dccf55b954b9a3d11f678249f00c35762137df276d
Sep 06 05:06:16.872511 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:16.872470    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2" event={"ID":"8bdaa50f-943f-4b93-8fa0-3787fab7191f","Type":"ContainerStarted","Data":"4dd7404569cb4589f1d04e4b635bf7fb9736c7fe7717ea041e078af42407b970"}
Sep 06 05:06:16.872971 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:16.872567    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"
Sep 06 05:06:16.873748 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:16.873722    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-zv545" event={"ID":"615696ff-75b3-4999-900a-087a50282695","Type":"ContainerStarted","Data":"3472cc9d381404d5c241a6dccf55b954b9a3d11f678249f00c35762137df276d"}
Sep 06 05:06:16.889543 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:16.889491    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2" podStartSLOduration=1.7259391339999999 podStartE2EDuration="2.889476275s" totalImagesPullingTime="1.163537141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:06:16.887732383 +0000 UTC m=+644.257187630" watchObservedRunningTime="2026-09-06 05:06:16.889476275 +0000 UTC m=+644.258931521"
Sep 06 05:06:18.859312 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:18.859278    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:18.886070 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:18.886028    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-zv545" event={"ID":"615696ff-75b3-4999-900a-087a50282695","Type":"ContainerStarted","Data":"508e6ebc59268a14c93d35cddbd30822a5cbe511d801fc4d5836e6f6c93c47a7"}
Sep 06 05:06:18.899458 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:18.899412    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-zv545" podStartSLOduration=2.020800349 podStartE2EDuration="3.899398276s" totalImagesPullingTime="1.878597927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:06:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:06:18.897878973 +0000 UTC m=+646.267334220" watchObservedRunningTime="2026-09-06 05:06:18.899398276 +0000 UTC m=+646.268853522"
Sep 06 05:06:18.961769 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:18.961737    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:19.259723 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:19.259682    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:19.313309 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:19.313273    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:19.663379 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:19.663346    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:19.763412 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:19.763372    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:20.164221 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:20.164189    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-zv545"]
Sep 06 05:06:20.896501 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:20.896467    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-zv545" podUID="615696ff-75b3-4999-900a-087a50282695" containerName="authorino" containerID="cri-o://508e6ebc59268a14c93d35cddbd30822a5cbe511d801fc4d5836e6f6c93c47a7" gracePeriod=30
Sep 06 05:06:21.146399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:21.146375    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-zv545"
Sep 06 05:06:21.268537 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:21.268458    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtln2\" (UniqueName: \"kubernetes.io/projected/615696ff-75b3-4999-900a-087a50282695-kube-api-access-jtln2\") pod \"615696ff-75b3-4999-900a-087a50282695\" (UID: \"615696ff-75b3-4999-900a-087a50282695\") "
Sep 06 05:06:21.270500 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:21.270473    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/615696ff-75b3-4999-900a-087a50282695-kube-api-access-jtln2" (OuterVolumeSpecName: "kube-api-access-jtln2") pod "615696ff-75b3-4999-900a-087a50282695" (UID: "615696ff-75b3-4999-900a-087a50282695"). InnerVolumeSpecName "kube-api-access-jtln2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:06:21.369501 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:21.369464    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtln2\" (UniqueName: \"kubernetes.io/projected/615696ff-75b3-4999-900a-087a50282695-kube-api-access-jtln2\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:06:21.903041 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:21.902984    2578 generic.go:356] "Generic (PLEG): container finished" podID="615696ff-75b3-4999-900a-087a50282695" containerID="508e6ebc59268a14c93d35cddbd30822a5cbe511d801fc4d5836e6f6c93c47a7" exitCode=0
Sep 06 05:06:21.903211 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:21.903063    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-zv545"
Sep 06 05:06:21.903211 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:21.903068    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-zv545" event={"ID":"615696ff-75b3-4999-900a-087a50282695","Type":"ContainerDied","Data":"508e6ebc59268a14c93d35cddbd30822a5cbe511d801fc4d5836e6f6c93c47a7"}
Sep 06 05:06:21.903211 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:21.903115    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-zv545" event={"ID":"615696ff-75b3-4999-900a-087a50282695","Type":"ContainerDied","Data":"3472cc9d381404d5c241a6dccf55b954b9a3d11f678249f00c35762137df276d"}
Sep 06 05:06:21.903211 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:21.903133    2578 scope.go:117] "RemoveContainer" containerID="508e6ebc59268a14c93d35cddbd30822a5cbe511d801fc4d5836e6f6c93c47a7"
Sep 06 05:06:21.911387 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:21.911368    2578 scope.go:117] "RemoveContainer" containerID="508e6ebc59268a14c93d35cddbd30822a5cbe511d801fc4d5836e6f6c93c47a7"
Sep 06 05:06:21.911635 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:06:21.911617    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"508e6ebc59268a14c93d35cddbd30822a5cbe511d801fc4d5836e6f6c93c47a7\": container with ID starting with 508e6ebc59268a14c93d35cddbd30822a5cbe511d801fc4d5836e6f6c93c47a7 not found: ID does not exist" containerID="508e6ebc59268a14c93d35cddbd30822a5cbe511d801fc4d5836e6f6c93c47a7"
Sep 06 05:06:21.911682 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:21.911643    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"508e6ebc59268a14c93d35cddbd30822a5cbe511d801fc4d5836e6f6c93c47a7"} err="failed to get container status \"508e6ebc59268a14c93d35cddbd30822a5cbe511d801fc4d5836e6f6c93c47a7\": rpc error: code = NotFound desc = could not find container \"508e6ebc59268a14c93d35cddbd30822a5cbe511d801fc4d5836e6f6c93c47a7\": container with ID starting with 508e6ebc59268a14c93d35cddbd30822a5cbe511d801fc4d5836e6f6c93c47a7 not found: ID does not exist"
Sep 06 05:06:21.921431 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:21.921409    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-zv545"]
Sep 06 05:06:21.926796 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:21.926775    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-zv545"]
Sep 06 05:06:23.152889 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:23.152855    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="615696ff-75b3-4999-900a-087a50282695" path="/var/lib/kubelet/pods/615696ff-75b3-4999-900a-087a50282695/volumes"
Sep 06 05:06:27.880717 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:27.880682    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"
Sep 06 05:06:28.074118 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:28.074083    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:28.171770 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:28.171682    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:30.507873 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:30.507837    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:30.560722 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:30.560688    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:31.112179 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.112140    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"]
Sep 06 05:06:31.112353 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.112320    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2" podUID="8bdaa50f-943f-4b93-8fa0-3787fab7191f" containerName="limitador" containerID="cri-o://4dd7404569cb4589f1d04e4b635bf7fb9736c7fe7717ea041e078af42407b970" gracePeriod=30
Sep 06 05:06:31.659055 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.659009    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"
Sep 06 05:06:31.756315 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.756228    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj4xd\" (UniqueName: \"kubernetes.io/projected/8bdaa50f-943f-4b93-8fa0-3787fab7191f-kube-api-access-vj4xd\") pod \"8bdaa50f-943f-4b93-8fa0-3787fab7191f\" (UID: \"8bdaa50f-943f-4b93-8fa0-3787fab7191f\") "
Sep 06 05:06:31.756470 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.756337    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8bdaa50f-943f-4b93-8fa0-3787fab7191f-config-file\") pod \"8bdaa50f-943f-4b93-8fa0-3787fab7191f\" (UID: \"8bdaa50f-943f-4b93-8fa0-3787fab7191f\") "
Sep 06 05:06:31.756691 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.756655    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bdaa50f-943f-4b93-8fa0-3787fab7191f-config-file" (OuterVolumeSpecName: "config-file") pod "8bdaa50f-943f-4b93-8fa0-3787fab7191f" (UID: "8bdaa50f-943f-4b93-8fa0-3787fab7191f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:06:31.758305 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.758284    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bdaa50f-943f-4b93-8fa0-3787fab7191f-kube-api-access-vj4xd" (OuterVolumeSpecName: "kube-api-access-vj4xd") pod "8bdaa50f-943f-4b93-8fa0-3787fab7191f" (UID: "8bdaa50f-943f-4b93-8fa0-3787fab7191f"). InnerVolumeSpecName "kube-api-access-vj4xd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:06:31.857328 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.857291    2578 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8bdaa50f-943f-4b93-8fa0-3787fab7191f-config-file\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:06:31.857328 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.857321    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj4xd\" (UniqueName: \"kubernetes.io/projected/8bdaa50f-943f-4b93-8fa0-3787fab7191f-kube-api-access-vj4xd\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:06:31.956001 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.955965    2578 generic.go:356] "Generic (PLEG): container finished" podID="8bdaa50f-943f-4b93-8fa0-3787fab7191f" containerID="4dd7404569cb4589f1d04e4b635bf7fb9736c7fe7717ea041e078af42407b970" exitCode=0
Sep 06 05:06:31.956187 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.956010    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"
Sep 06 05:06:31.956187 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.956051    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2" event={"ID":"8bdaa50f-943f-4b93-8fa0-3787fab7191f","Type":"ContainerDied","Data":"4dd7404569cb4589f1d04e4b635bf7fb9736c7fe7717ea041e078af42407b970"}
Sep 06 05:06:31.956187 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.956111    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-7h4z2" event={"ID":"8bdaa50f-943f-4b93-8fa0-3787fab7191f","Type":"ContainerDied","Data":"043491c9d894de102fc48b2dc08a03ecdcd4de8435660d375f9ce7fa759fa731"}
Sep 06 05:06:31.956187 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.956135    2578 scope.go:117] "RemoveContainer" containerID="4dd7404569cb4589f1d04e4b635bf7fb9736c7fe7717ea041e078af42407b970"
Sep 06 05:06:31.965251 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.965234    2578 scope.go:117] "RemoveContainer" containerID="4dd7404569cb4589f1d04e4b635bf7fb9736c7fe7717ea041e078af42407b970"
Sep 06 05:06:31.965492 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:06:31.965473    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dd7404569cb4589f1d04e4b635bf7fb9736c7fe7717ea041e078af42407b970\": container with ID starting with 4dd7404569cb4589f1d04e4b635bf7fb9736c7fe7717ea041e078af42407b970 not found: ID does not exist" containerID="4dd7404569cb4589f1d04e4b635bf7fb9736c7fe7717ea041e078af42407b970"
Sep 06 05:06:31.965547 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.965500    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dd7404569cb4589f1d04e4b635bf7fb9736c7fe7717ea041e078af42407b970"} err="failed to get container status \"4dd7404569cb4589f1d04e4b635bf7fb9736c7fe7717ea041e078af42407b970\": rpc error: code = NotFound desc = could not find container \"4dd7404569cb4589f1d04e4b635bf7fb9736c7fe7717ea041e078af42407b970\": container with ID starting with 4dd7404569cb4589f1d04e4b635bf7fb9736c7fe7717ea041e078af42407b970 not found: ID does not exist"
Sep 06 05:06:31.975765 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.975741    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"]
Sep 06 05:06:31.978779 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:31.978759    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7h4z2"]
Sep 06 05:06:33.158168 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:33.154877    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bdaa50f-943f-4b93-8fa0-3787fab7191f" path="/var/lib/kubelet/pods/8bdaa50f-943f-4b93-8fa0-3787fab7191f/volumes"
Sep 06 05:06:33.898778 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:33.898740    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-69fdc6d5bd-b4dmg" podUID="3be4d2db-df80-4f74-8297-3b2428250eaa" containerName="console" containerID="cri-o://67c3d6e64c6e6f0e5508d088c10cc94c42523e9f099f3ef3499f34e33f1b1ee1" gracePeriod=15
Sep 06 05:06:34.139788 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.139765    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:06:34.279522 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.279433    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be4d2db-df80-4f74-8297-3b2428250eaa-console-serving-cert\") pod \"3be4d2db-df80-4f74-8297-3b2428250eaa\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") "
Sep 06 05:06:34.279522 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.279473    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-service-ca\") pod \"3be4d2db-df80-4f74-8297-3b2428250eaa\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") "
Sep 06 05:06:34.279522 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.279508    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28x5z\" (UniqueName: \"kubernetes.io/projected/3be4d2db-df80-4f74-8297-3b2428250eaa-kube-api-access-28x5z\") pod \"3be4d2db-df80-4f74-8297-3b2428250eaa\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") "
Sep 06 05:06:34.280075 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.279552    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-trusted-ca-bundle\") pod \"3be4d2db-df80-4f74-8297-3b2428250eaa\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") "
Sep 06 05:06:34.280075 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.279569    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-oauth-serving-cert\") pod \"3be4d2db-df80-4f74-8297-3b2428250eaa\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") "
Sep 06 05:06:34.280075 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.279583    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-console-config\") pod \"3be4d2db-df80-4f74-8297-3b2428250eaa\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") "
Sep 06 05:06:34.280075 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.279637    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be4d2db-df80-4f74-8297-3b2428250eaa-console-oauth-config\") pod \"3be4d2db-df80-4f74-8297-3b2428250eaa\" (UID: \"3be4d2db-df80-4f74-8297-3b2428250eaa\") "
Sep 06 05:06:34.280302 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.280068    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-service-ca" (OuterVolumeSpecName: "service-ca") pod "3be4d2db-df80-4f74-8297-3b2428250eaa" (UID: "3be4d2db-df80-4f74-8297-3b2428250eaa"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:06:34.280302 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.280089    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3be4d2db-df80-4f74-8297-3b2428250eaa" (UID: "3be4d2db-df80-4f74-8297-3b2428250eaa"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:06:34.280302 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.280147    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3be4d2db-df80-4f74-8297-3b2428250eaa" (UID: "3be4d2db-df80-4f74-8297-3b2428250eaa"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:06:34.280302 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.280196    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-console-config" (OuterVolumeSpecName: "console-config") pod "3be4d2db-df80-4f74-8297-3b2428250eaa" (UID: "3be4d2db-df80-4f74-8297-3b2428250eaa"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:06:34.281708 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.281681    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3be4d2db-df80-4f74-8297-3b2428250eaa-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3be4d2db-df80-4f74-8297-3b2428250eaa" (UID: "3be4d2db-df80-4f74-8297-3b2428250eaa"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:06:34.281807 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.281747    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3be4d2db-df80-4f74-8297-3b2428250eaa-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3be4d2db-df80-4f74-8297-3b2428250eaa" (UID: "3be4d2db-df80-4f74-8297-3b2428250eaa"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:06:34.281870 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.281854    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3be4d2db-df80-4f74-8297-3b2428250eaa-kube-api-access-28x5z" (OuterVolumeSpecName: "kube-api-access-28x5z") pod "3be4d2db-df80-4f74-8297-3b2428250eaa" (UID: "3be4d2db-df80-4f74-8297-3b2428250eaa"). InnerVolumeSpecName "kube-api-access-28x5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:06:34.381191 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.381159    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28x5z\" (UniqueName: \"kubernetes.io/projected/3be4d2db-df80-4f74-8297-3b2428250eaa-kube-api-access-28x5z\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:06:34.381191 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.381188    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-trusted-ca-bundle\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:06:34.381191 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.381198    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-oauth-serving-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:06:34.381414 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.381206    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-console-config\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:06:34.381414 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.381216    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be4d2db-df80-4f74-8297-3b2428250eaa-console-oauth-config\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:06:34.381414 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.381224    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be4d2db-df80-4f74-8297-3b2428250eaa-console-serving-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:06:34.381414 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.381233    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be4d2db-df80-4f74-8297-3b2428250eaa-service-ca\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:06:34.974104 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.974071    2578 generic.go:356] "Generic (PLEG): container finished" podID="3be4d2db-df80-4f74-8297-3b2428250eaa" containerID="67c3d6e64c6e6f0e5508d088c10cc94c42523e9f099f3ef3499f34e33f1b1ee1" exitCode=0
Sep 06 05:06:34.974273 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.974119    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69fdc6d5bd-b4dmg"
Sep 06 05:06:34.974273 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.974153    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69fdc6d5bd-b4dmg" event={"ID":"3be4d2db-df80-4f74-8297-3b2428250eaa","Type":"ContainerDied","Data":"67c3d6e64c6e6f0e5508d088c10cc94c42523e9f099f3ef3499f34e33f1b1ee1"}
Sep 06 05:06:34.974273 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.974204    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69fdc6d5bd-b4dmg" event={"ID":"3be4d2db-df80-4f74-8297-3b2428250eaa","Type":"ContainerDied","Data":"acda2c7e22541fb055bd1892a1ac1538ad00104393c32178954b6e27b43ede7f"}
Sep 06 05:06:34.974273 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.974225    2578 scope.go:117] "RemoveContainer" containerID="67c3d6e64c6e6f0e5508d088c10cc94c42523e9f099f3ef3499f34e33f1b1ee1"
Sep 06 05:06:34.983344 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.983326    2578 scope.go:117] "RemoveContainer" containerID="67c3d6e64c6e6f0e5508d088c10cc94c42523e9f099f3ef3499f34e33f1b1ee1"
Sep 06 05:06:34.983622 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:06:34.983604    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67c3d6e64c6e6f0e5508d088c10cc94c42523e9f099f3ef3499f34e33f1b1ee1\": container with ID starting with 67c3d6e64c6e6f0e5508d088c10cc94c42523e9f099f3ef3499f34e33f1b1ee1 not found: ID does not exist" containerID="67c3d6e64c6e6f0e5508d088c10cc94c42523e9f099f3ef3499f34e33f1b1ee1"
Sep 06 05:06:34.983688 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.983631    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67c3d6e64c6e6f0e5508d088c10cc94c42523e9f099f3ef3499f34e33f1b1ee1"} err="failed to get container status \"67c3d6e64c6e6f0e5508d088c10cc94c42523e9f099f3ef3499f34e33f1b1ee1\": rpc error: code = NotFound desc = could not find container \"67c3d6e64c6e6f0e5508d088c10cc94c42523e9f099f3ef3499f34e33f1b1ee1\": container with ID starting with 67c3d6e64c6e6f0e5508d088c10cc94c42523e9f099f3ef3499f34e33f1b1ee1 not found: ID does not exist"
Sep 06 05:06:34.995190 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.995161    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69fdc6d5bd-b4dmg"]
Sep 06 05:06:34.996721 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:34.996704    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69fdc6d5bd-b4dmg"]
Sep 06 05:06:35.152717 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:35.152685    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3be4d2db-df80-4f74-8297-3b2428250eaa" path="/var/lib/kubelet/pods/3be4d2db-df80-4f74-8297-3b2428250eaa/volumes"
Sep 06 05:06:35.272585 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:35.272503    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:35.373347 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:35.373308    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:36.154204 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:36.154170    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:36.254586 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:36.254553    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:36.632795 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:36.632764    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:48.309222 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.309187    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:48.525830 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.525797    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-w8275"]
Sep 06 05:06:48.526171 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.526155    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3be4d2db-df80-4f74-8297-3b2428250eaa" containerName="console"
Sep 06 05:06:48.526228 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.526175    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3be4d2db-df80-4f74-8297-3b2428250eaa" containerName="console"
Sep 06 05:06:48.526228 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.526198    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="615696ff-75b3-4999-900a-087a50282695" containerName="authorino"
Sep 06 05:06:48.526228 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.526204    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="615696ff-75b3-4999-900a-087a50282695" containerName="authorino"
Sep 06 05:06:48.526228 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.526220    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bdaa50f-943f-4b93-8fa0-3787fab7191f" containerName="limitador"
Sep 06 05:06:48.526228 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.526227    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bdaa50f-943f-4b93-8fa0-3787fab7191f" containerName="limitador"
Sep 06 05:06:48.526392 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.526282    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3be4d2db-df80-4f74-8297-3b2428250eaa" containerName="console"
Sep 06 05:06:48.526392 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.526290    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bdaa50f-943f-4b93-8fa0-3787fab7191f" containerName="limitador"
Sep 06 05:06:48.526392 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.526298    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="615696ff-75b3-4999-900a-087a50282695" containerName="authorino"
Sep 06 05:06:48.532178 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.532151    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-w8275"
Sep 06 05:06:48.534504 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.534479    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-w8275"]
Sep 06 05:06:48.534504 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.534501    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-phjph\""
Sep 06 05:06:48.709454 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.709424    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkdkq\" (UniqueName: \"kubernetes.io/projected/e27a63ee-20a5-414c-8b81-4db7c2852ab5-kube-api-access-rkdkq\") pod \"authorino-5dc7ccb5f-w8275\" (UID: \"e27a63ee-20a5-414c-8b81-4db7c2852ab5\") " pod="kuadrant-system/authorino-5dc7ccb5f-w8275"
Sep 06 05:06:48.755609 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.755574    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-w8275"]
Sep 06 05:06:48.755846 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:06:48.755827    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rkdkq], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-w8275" podUID="e27a63ee-20a5-414c-8b81-4db7c2852ab5"
Sep 06 05:06:48.784545 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.784513    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7996f5d69d-2zxd2"]
Sep 06 05:06:48.788644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.788617    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7996f5d69d-2zxd2"
Sep 06 05:06:48.795608 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.795580    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7996f5d69d-2zxd2"]
Sep 06 05:06:48.799333 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.799307    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:48.810349 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.810327    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkdkq\" (UniqueName: \"kubernetes.io/projected/e27a63ee-20a5-414c-8b81-4db7c2852ab5-kube-api-access-rkdkq\") pod \"authorino-5dc7ccb5f-w8275\" (UID: \"e27a63ee-20a5-414c-8b81-4db7c2852ab5\") " pod="kuadrant-system/authorino-5dc7ccb5f-w8275"
Sep 06 05:06:48.817315 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.817294    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkdkq\" (UniqueName: \"kubernetes.io/projected/e27a63ee-20a5-414c-8b81-4db7c2852ab5-kube-api-access-rkdkq\") pod \"authorino-5dc7ccb5f-w8275\" (UID: \"e27a63ee-20a5-414c-8b81-4db7c2852ab5\") " pod="kuadrant-system/authorino-5dc7ccb5f-w8275"
Sep 06 05:06:48.857228 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.857190    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7996f5d69d-2zxd2"]
Sep 06 05:06:48.857462 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:06:48.857444    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-kzzp7], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7996f5d69d-2zxd2" podUID="123fea25-b815-46da-9b49-d0423b1b34ba"
Sep 06 05:06:48.887423 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.887392    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c5fbddd7b-zvn8z"]
Sep 06 05:06:48.891393 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.891372    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c5fbddd7b-zvn8z"
Sep 06 05:06:48.893693 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.893676    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 06 05:06:48.899940 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.899920    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c5fbddd7b-zvn8z"]
Sep 06 05:06:48.911269 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:48.911240    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzzp7\" (UniqueName: \"kubernetes.io/projected/123fea25-b815-46da-9b49-d0423b1b34ba-kube-api-access-kzzp7\") pod \"authorino-7996f5d69d-2zxd2\" (UID: \"123fea25-b815-46da-9b49-d0423b1b34ba\") " pod="kuadrant-system/authorino-7996f5d69d-2zxd2"
Sep 06 05:06:49.001097 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.000979    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:49.012399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.012370    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ee71311-c437-4c43-b4a5-bdeb974f36bb-tls-cert\") pod \"authorino-c5fbddd7b-zvn8z\" (UID: \"5ee71311-c437-4c43-b4a5-bdeb974f36bb\") " pod="kuadrant-system/authorino-c5fbddd7b-zvn8z"
Sep 06 05:06:49.012509 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.012404    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwpnm\" (UniqueName: \"kubernetes.io/projected/5ee71311-c437-4c43-b4a5-bdeb974f36bb-kube-api-access-mwpnm\") pod \"authorino-c5fbddd7b-zvn8z\" (UID: \"5ee71311-c437-4c43-b4a5-bdeb974f36bb\") " pod="kuadrant-system/authorino-c5fbddd7b-zvn8z"
Sep 06 05:06:49.012549 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.012515    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzzp7\" (UniqueName: \"kubernetes.io/projected/123fea25-b815-46da-9b49-d0423b1b34ba-kube-api-access-kzzp7\") pod \"authorino-7996f5d69d-2zxd2\" (UID: \"123fea25-b815-46da-9b49-d0423b1b34ba\") " pod="kuadrant-system/authorino-7996f5d69d-2zxd2"
Sep 06 05:06:49.021897 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.021870    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzzp7\" (UniqueName: \"kubernetes.io/projected/123fea25-b815-46da-9b49-d0423b1b34ba-kube-api-access-kzzp7\") pod \"authorino-7996f5d69d-2zxd2\" (UID: \"123fea25-b815-46da-9b49-d0423b1b34ba\") " pod="kuadrant-system/authorino-7996f5d69d-2zxd2"
Sep 06 05:06:49.038574 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.038546    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-w8275"
Sep 06 05:06:49.038682 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.038546    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7996f5d69d-2zxd2"
Sep 06 05:06:49.046165 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.046145    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-w8275"
Sep 06 05:06:49.049046 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.049007    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7996f5d69d-2zxd2"
Sep 06 05:06:49.113653 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.113620    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ee71311-c437-4c43-b4a5-bdeb974f36bb-tls-cert\") pod \"authorino-c5fbddd7b-zvn8z\" (UID: \"5ee71311-c437-4c43-b4a5-bdeb974f36bb\") " pod="kuadrant-system/authorino-c5fbddd7b-zvn8z"
Sep 06 05:06:49.113653 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.113654    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwpnm\" (UniqueName: \"kubernetes.io/projected/5ee71311-c437-4c43-b4a5-bdeb974f36bb-kube-api-access-mwpnm\") pod \"authorino-c5fbddd7b-zvn8z\" (UID: \"5ee71311-c437-4c43-b4a5-bdeb974f36bb\") " pod="kuadrant-system/authorino-c5fbddd7b-zvn8z"
Sep 06 05:06:49.115975 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.115953    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ee71311-c437-4c43-b4a5-bdeb974f36bb-tls-cert\") pod \"authorino-c5fbddd7b-zvn8z\" (UID: \"5ee71311-c437-4c43-b4a5-bdeb974f36bb\") " pod="kuadrant-system/authorino-c5fbddd7b-zvn8z"
Sep 06 05:06:49.120731 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.120705    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwpnm\" (UniqueName: \"kubernetes.io/projected/5ee71311-c437-4c43-b4a5-bdeb974f36bb-kube-api-access-mwpnm\") pod \"authorino-c5fbddd7b-zvn8z\" (UID: \"5ee71311-c437-4c43-b4a5-bdeb974f36bb\") " pod="kuadrant-system/authorino-c5fbddd7b-zvn8z"
Sep 06 05:06:49.201639 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.201605    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c5fbddd7b-zvn8z"
Sep 06 05:06:49.214999 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.214975    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkdkq\" (UniqueName: \"kubernetes.io/projected/e27a63ee-20a5-414c-8b81-4db7c2852ab5-kube-api-access-rkdkq\") pod \"e27a63ee-20a5-414c-8b81-4db7c2852ab5\" (UID: \"e27a63ee-20a5-414c-8b81-4db7c2852ab5\") "
Sep 06 05:06:49.215193 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.215173    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzzp7\" (UniqueName: \"kubernetes.io/projected/123fea25-b815-46da-9b49-d0423b1b34ba-kube-api-access-kzzp7\") pod \"123fea25-b815-46da-9b49-d0423b1b34ba\" (UID: \"123fea25-b815-46da-9b49-d0423b1b34ba\") "
Sep 06 05:06:49.217334 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.217310    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e27a63ee-20a5-414c-8b81-4db7c2852ab5-kube-api-access-rkdkq" (OuterVolumeSpecName: "kube-api-access-rkdkq") pod "e27a63ee-20a5-414c-8b81-4db7c2852ab5" (UID: "e27a63ee-20a5-414c-8b81-4db7c2852ab5"). InnerVolumeSpecName "kube-api-access-rkdkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:06:49.217408 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.217356    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/123fea25-b815-46da-9b49-d0423b1b34ba-kube-api-access-kzzp7" (OuterVolumeSpecName: "kube-api-access-kzzp7") pod "123fea25-b815-46da-9b49-d0423b1b34ba" (UID: "123fea25-b815-46da-9b49-d0423b1b34ba"). InnerVolumeSpecName "kube-api-access-kzzp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:06:49.316001 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.315910    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzzp7\" (UniqueName: \"kubernetes.io/projected/123fea25-b815-46da-9b49-d0423b1b34ba-kube-api-access-kzzp7\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:06:49.316001 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.315944    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkdkq\" (UniqueName: \"kubernetes.io/projected/e27a63ee-20a5-414c-8b81-4db7c2852ab5-kube-api-access-rkdkq\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:06:49.468914 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:49.468888    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c5fbddd7b-zvn8z"]
Sep 06 05:06:49.471101 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:06:49.471078    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ee71311_c437_4c43_b4a5_bdeb974f36bb.slice/crio-f5f3a6b9642e2f0f20d4fee1535532dc8de876254aabd070184aa0698b0013d8 WatchSource:0}: Error finding container f5f3a6b9642e2f0f20d4fee1535532dc8de876254aabd070184aa0698b0013d8: Status 404 returned error can't find the container with id f5f3a6b9642e2f0f20d4fee1535532dc8de876254aabd070184aa0698b0013d8
Sep 06 05:06:50.043942 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:50.043910    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c5fbddd7b-zvn8z" event={"ID":"5ee71311-c437-4c43-b4a5-bdeb974f36bb","Type":"ContainerStarted","Data":"f5f3a6b9642e2f0f20d4fee1535532dc8de876254aabd070184aa0698b0013d8"}
Sep 06 05:06:50.044128 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:50.043967    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7996f5d69d-2zxd2"
Sep 06 05:06:50.044128 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:50.043972    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-w8275"
Sep 06 05:06:50.070973 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:50.070947    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-w8275"]
Sep 06 05:06:50.074198 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:50.074143    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-w8275"]
Sep 06 05:06:50.095231 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:50.095199    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7996f5d69d-2zxd2"]
Sep 06 05:06:50.096583 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:50.096557    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7996f5d69d-2zxd2"]
Sep 06 05:06:51.051461 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:51.051429    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c5fbddd7b-zvn8z" event={"ID":"5ee71311-c437-4c43-b4a5-bdeb974f36bb","Type":"ContainerStarted","Data":"21b71a5536a10a31f0b374e7ef2975c2f367cf53dae917fc5f6eecbc643cd2fd"}
Sep 06 05:06:51.064701 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:51.064656    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c5fbddd7b-zvn8z" podStartSLOduration=2.292463908 podStartE2EDuration="3.064628683s" totalImagesPullingTime="772.164775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:06:51.063781222 +0000 UTC m=+678.433236469" watchObservedRunningTime="2026-09-06 05:06:51.064628683 +0000 UTC m=+678.434083929"
Sep 06 05:06:51.153336 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:51.153307    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="123fea25-b815-46da-9b49-d0423b1b34ba" path="/var/lib/kubelet/pods/123fea25-b815-46da-9b49-d0423b1b34ba/volumes"
Sep 06 05:06:51.153535 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:51.153524    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e27a63ee-20a5-414c-8b81-4db7c2852ab5" path="/var/lib/kubelet/pods/e27a63ee-20a5-414c-8b81-4db7c2852ab5/volumes"
Sep 06 05:06:55.840680 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:55.840636    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:55.863947 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:55.863912    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67b48bfc7b-gmb2f"]
Sep 06 05:06:55.867558 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:55.867536    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:06:55.869766 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:55.869743    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 06 05:06:55.869766 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:55.869758    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 06 05:06:55.870641 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:55.870619    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vpcfl\""
Sep 06 05:06:55.870742 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:55.870642    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 06 05:06:55.878918 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:55.878899    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b48bfc7b-gmb2f"]
Sep 06 05:06:55.939614 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:55.939577    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:55.973830 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:55.973797    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-metrics-certs\") pod \"maas-controller-67b48bfc7b-gmb2f\" (UID: \"10a3daff-f870-489f-ab74-9d15085020e5\") " pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:06:55.973984 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:55.973848    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-webhook-cert\") pod \"maas-controller-67b48bfc7b-gmb2f\" (UID: \"10a3daff-f870-489f-ab74-9d15085020e5\") " pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:06:55.973984 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:55.973863    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nh65\" (UniqueName: \"kubernetes.io/projected/10a3daff-f870-489f-ab74-9d15085020e5-kube-api-access-5nh65\") pod \"maas-controller-67b48bfc7b-gmb2f\" (UID: \"10a3daff-f870-489f-ab74-9d15085020e5\") " pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:06:56.040865 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.040831    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:56.047075 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.047047    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-q9kvl"]
Sep 06 05:06:56.050317 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.050297    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:06:56.060902 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.060879    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-q9kvl"]
Sep 06 05:06:56.074725 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.074699    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-webhook-cert\") pod \"maas-controller-67b48bfc7b-gmb2f\" (UID: \"10a3daff-f870-489f-ab74-9d15085020e5\") " pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:06:56.074872 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.074732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nh65\" (UniqueName: \"kubernetes.io/projected/10a3daff-f870-489f-ab74-9d15085020e5-kube-api-access-5nh65\") pod \"maas-controller-67b48bfc7b-gmb2f\" (UID: \"10a3daff-f870-489f-ab74-9d15085020e5\") " pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:06:56.074872 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.074804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-metrics-certs\") pod \"maas-controller-67b48bfc7b-gmb2f\" (UID: \"10a3daff-f870-489f-ab74-9d15085020e5\") " pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:06:56.075011 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:06:56.074916    2578 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 06 05:06:56.075011 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:06:56.074967    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-metrics-certs podName:10a3daff-f870-489f-ab74-9d15085020e5 nodeName:}" failed. No retries permitted until 2026-09-06 05:06:56.574950044 +0000 UTC m=+683.944405268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-metrics-certs") pod "maas-controller-67b48bfc7b-gmb2f" (UID: "10a3daff-f870-489f-ab74-9d15085020e5") : secret "maas-controller-metrics-tls" not found
Sep 06 05:06:56.077171 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.077150    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-webhook-cert\") pod \"maas-controller-67b48bfc7b-gmb2f\" (UID: \"10a3daff-f870-489f-ab74-9d15085020e5\") " pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:06:56.084736 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.084700    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nh65\" (UniqueName: \"kubernetes.io/projected/10a3daff-f870-489f-ab74-9d15085020e5-kube-api-access-5nh65\") pod \"maas-controller-67b48bfc7b-gmb2f\" (UID: \"10a3daff-f870-489f-ab74-9d15085020e5\") " pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:06:56.139877 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.139844    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:56.175420 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.175391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-metrics-certs\") pod \"maas-controller-8557bfdbbd-q9kvl\" (UID: \"0c7370b3-6b61-428b-89fc-ebd9162c90da\") " pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:06:56.175553 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.175436    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-webhook-cert\") pod \"maas-controller-8557bfdbbd-q9kvl\" (UID: \"0c7370b3-6b61-428b-89fc-ebd9162c90da\") " pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:06:56.175553 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.175496    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stwml\" (UniqueName: \"kubernetes.io/projected/0c7370b3-6b61-428b-89fc-ebd9162c90da-kube-api-access-stwml\") pod \"maas-controller-8557bfdbbd-q9kvl\" (UID: \"0c7370b3-6b61-428b-89fc-ebd9162c90da\") " pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:06:56.276932 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.276896    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-metrics-certs\") pod \"maas-controller-8557bfdbbd-q9kvl\" (UID: \"0c7370b3-6b61-428b-89fc-ebd9162c90da\") " pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:06:56.277118 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.276944    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-webhook-cert\") pod \"maas-controller-8557bfdbbd-q9kvl\" (UID: \"0c7370b3-6b61-428b-89fc-ebd9162c90da\") " pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:06:56.277118 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.276999    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stwml\" (UniqueName: \"kubernetes.io/projected/0c7370b3-6b61-428b-89fc-ebd9162c90da-kube-api-access-stwml\") pod \"maas-controller-8557bfdbbd-q9kvl\" (UID: \"0c7370b3-6b61-428b-89fc-ebd9162c90da\") " pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:06:56.277118 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:06:56.277094    2578 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 06 05:06:56.277276 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:06:56.277165    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-metrics-certs podName:0c7370b3-6b61-428b-89fc-ebd9162c90da nodeName:}" failed. No retries permitted until 2026-09-06 05:06:56.777143441 +0000 UTC m=+684.146598684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-metrics-certs") pod "maas-controller-8557bfdbbd-q9kvl" (UID: "0c7370b3-6b61-428b-89fc-ebd9162c90da") : secret "maas-controller-metrics-tls" not found
Sep 06 05:06:56.279392 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.279373    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-webhook-cert\") pod \"maas-controller-8557bfdbbd-q9kvl\" (UID: \"0c7370b3-6b61-428b-89fc-ebd9162c90da\") " pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:06:56.284963 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.284939    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stwml\" (UniqueName: \"kubernetes.io/projected/0c7370b3-6b61-428b-89fc-ebd9162c90da-kube-api-access-stwml\") pod \"maas-controller-8557bfdbbd-q9kvl\" (UID: \"0c7370b3-6b61-428b-89fc-ebd9162c90da\") " pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:06:56.325580 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.325550    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:56.579384 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.579298    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-metrics-certs\") pod \"maas-controller-67b48bfc7b-gmb2f\" (UID: \"10a3daff-f870-489f-ab74-9d15085020e5\") " pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:06:56.581626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.581603    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-metrics-certs\") pod \"maas-controller-67b48bfc7b-gmb2f\" (UID: \"10a3daff-f870-489f-ab74-9d15085020e5\") " pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:06:56.778285 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.778251    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:06:56.780597 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.780573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-metrics-certs\") pod \"maas-controller-8557bfdbbd-q9kvl\" (UID: \"0c7370b3-6b61-428b-89fc-ebd9162c90da\") " pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:06:56.782925 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.782899    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-metrics-certs\") pod \"maas-controller-8557bfdbbd-q9kvl\" (UID: \"0c7370b3-6b61-428b-89fc-ebd9162c90da\") " pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:06:56.961552 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:56.961518    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:06:57.048837 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:57.048813    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b48bfc7b-gmb2f"]
Sep 06 05:06:57.050709 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:06:57.050680    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a3daff_f870_489f_ab74_9d15085020e5.slice/crio-9a01dee369e454eb892117a9e13e9999eb887c514feccd3184f57a32965b735c WatchSource:0}: Error finding container 9a01dee369e454eb892117a9e13e9999eb887c514feccd3184f57a32965b735c: Status 404 returned error can't find the container with id 9a01dee369e454eb892117a9e13e9999eb887c514feccd3184f57a32965b735c
Sep 06 05:06:57.082777 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:57.082743    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b48bfc7b-gmb2f" event={"ID":"10a3daff-f870-489f-ab74-9d15085020e5","Type":"ContainerStarted","Data":"9a01dee369e454eb892117a9e13e9999eb887c514feccd3184f57a32965b735c"}
Sep 06 05:06:57.230569 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:57.230546    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-q9kvl"]
Sep 06 05:06:57.232877 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:06:57.232847    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c7370b3_6b61_428b_89fc_ebd9162c90da.slice/crio-f89569bec6da7086311032f2dbf9ef429bdd85647c3b956dc9d2ecb06ffe9e9e WatchSource:0}: Error finding container f89569bec6da7086311032f2dbf9ef429bdd85647c3b956dc9d2ecb06ffe9e9e: Status 404 returned error can't find the container with id f89569bec6da7086311032f2dbf9ef429bdd85647c3b956dc9d2ecb06ffe9e9e
Sep 06 05:06:58.097364 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:58.097326    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8557bfdbbd-q9kvl" event={"ID":"0c7370b3-6b61-428b-89fc-ebd9162c90da","Type":"ContainerStarted","Data":"f89569bec6da7086311032f2dbf9ef429bdd85647c3b956dc9d2ecb06ffe9e9e"}
Sep 06 05:06:59.874394 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:59.874278    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:06:59.997602 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:06:59.997568    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:00.109948 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:00.109873    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8557bfdbbd-q9kvl" event={"ID":"0c7370b3-6b61-428b-89fc-ebd9162c90da","Type":"ContainerStarted","Data":"fd8bb2125610fae061f9024878c3166861f4721f845cb8e709b831f61a3288d6"}
Sep 06 05:07:00.110181 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:00.110000    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:07:00.111786 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:00.111758    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b48bfc7b-gmb2f" event={"ID":"10a3daff-f870-489f-ab74-9d15085020e5","Type":"ContainerStarted","Data":"3ceeaac8454b201c47e4a5ec56b40e8abde8b5d86b2b762592754025c1fa7534"}
Sep 06 05:07:00.111929 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:00.111912    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:07:00.132767 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:00.132630    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8557bfdbbd-q9kvl" podStartSLOduration=1.722668093 podStartE2EDuration="4.132607698s" totalImagesPullingTime="2.409939605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:07:00.131788893 +0000 UTC m=+687.501244155" watchObservedRunningTime="2026-09-06 05:07:00.132607698 +0000 UTC m=+687.502062925"
Sep 06 05:07:00.151040 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:00.150036    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67b48bfc7b-gmb2f" podStartSLOduration=2.570548228 podStartE2EDuration="5.149998048s" totalImagesPullingTime="2.57944982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:07:00.148701773 +0000 UTC m=+687.518157020" watchObservedRunningTime="2026-09-06 05:07:00.149998048 +0000 UTC m=+687.519453295"
Sep 06 05:07:06.303846 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:06.303801    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:07.947365 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:07.947333    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:11.120423 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:11.120381    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:07:11.120824 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:11.120456    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:07:11.177032 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:11.176980    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b48bfc7b-gmb2f"]
Sep 06 05:07:11.177469 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:11.177434    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67b48bfc7b-gmb2f" podUID="10a3daff-f870-489f-ab74-9d15085020e5" containerName="manager" containerID="cri-o://3ceeaac8454b201c47e4a5ec56b40e8abde8b5d86b2b762592754025c1fa7534" gracePeriod=30
Sep 06 05:07:11.424103 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:11.424078    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:07:11.508338 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:11.508302    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-webhook-cert\") pod \"10a3daff-f870-489f-ab74-9d15085020e5\" (UID: \"10a3daff-f870-489f-ab74-9d15085020e5\") "
Sep 06 05:07:11.508541 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:11.508392    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-metrics-certs\") pod \"10a3daff-f870-489f-ab74-9d15085020e5\" (UID: \"10a3daff-f870-489f-ab74-9d15085020e5\") "
Sep 06 05:07:11.508619 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:11.508551    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nh65\" (UniqueName: \"kubernetes.io/projected/10a3daff-f870-489f-ab74-9d15085020e5-kube-api-access-5nh65\") pod \"10a3daff-f870-489f-ab74-9d15085020e5\" (UID: \"10a3daff-f870-489f-ab74-9d15085020e5\") "
Sep 06 05:07:11.510618 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:11.510584    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "10a3daff-f870-489f-ab74-9d15085020e5" (UID: "10a3daff-f870-489f-ab74-9d15085020e5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:07:11.510618 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:11.510611    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "10a3daff-f870-489f-ab74-9d15085020e5" (UID: "10a3daff-f870-489f-ab74-9d15085020e5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:07:11.510779 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:11.510690    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10a3daff-f870-489f-ab74-9d15085020e5-kube-api-access-5nh65" (OuterVolumeSpecName: "kube-api-access-5nh65") pod "10a3daff-f870-489f-ab74-9d15085020e5" (UID: "10a3daff-f870-489f-ab74-9d15085020e5"). InnerVolumeSpecName "kube-api-access-5nh65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:07:11.609898 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:11.609857    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nh65\" (UniqueName: \"kubernetes.io/projected/10a3daff-f870-489f-ab74-9d15085020e5-kube-api-access-5nh65\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:07:11.609898 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:11.609893    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-webhook-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:07:11.609898 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:11.609907    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10a3daff-f870-489f-ab74-9d15085020e5-metrics-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:07:12.170320 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:12.170281    2578 generic.go:356] "Generic (PLEG): container finished" podID="10a3daff-f870-489f-ab74-9d15085020e5" containerID="3ceeaac8454b201c47e4a5ec56b40e8abde8b5d86b2b762592754025c1fa7534" exitCode=0
Sep 06 05:07:12.170758 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:12.170327    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b48bfc7b-gmb2f"
Sep 06 05:07:12.170758 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:12.170370    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b48bfc7b-gmb2f" event={"ID":"10a3daff-f870-489f-ab74-9d15085020e5","Type":"ContainerDied","Data":"3ceeaac8454b201c47e4a5ec56b40e8abde8b5d86b2b762592754025c1fa7534"}
Sep 06 05:07:12.170758 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:12.170419    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b48bfc7b-gmb2f" event={"ID":"10a3daff-f870-489f-ab74-9d15085020e5","Type":"ContainerDied","Data":"9a01dee369e454eb892117a9e13e9999eb887c514feccd3184f57a32965b735c"}
Sep 06 05:07:12.170758 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:12.170440    2578 scope.go:117] "RemoveContainer" containerID="3ceeaac8454b201c47e4a5ec56b40e8abde8b5d86b2b762592754025c1fa7534"
Sep 06 05:07:12.179767 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:12.179594    2578 scope.go:117] "RemoveContainer" containerID="3ceeaac8454b201c47e4a5ec56b40e8abde8b5d86b2b762592754025c1fa7534"
Sep 06 05:07:12.179860 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:07:12.179841    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ceeaac8454b201c47e4a5ec56b40e8abde8b5d86b2b762592754025c1fa7534\": container with ID starting with 3ceeaac8454b201c47e4a5ec56b40e8abde8b5d86b2b762592754025c1fa7534 not found: ID does not exist" containerID="3ceeaac8454b201c47e4a5ec56b40e8abde8b5d86b2b762592754025c1fa7534"
Sep 06 05:07:12.179899 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:12.179867    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ceeaac8454b201c47e4a5ec56b40e8abde8b5d86b2b762592754025c1fa7534"} err="failed to get container status \"3ceeaac8454b201c47e4a5ec56b40e8abde8b5d86b2b762592754025c1fa7534\": rpc error: code = NotFound desc = could not find container \"3ceeaac8454b201c47e4a5ec56b40e8abde8b5d86b2b762592754025c1fa7534\": container with ID starting with 3ceeaac8454b201c47e4a5ec56b40e8abde8b5d86b2b762592754025c1fa7534 not found: ID does not exist"
Sep 06 05:07:12.190191 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:12.190165    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b48bfc7b-gmb2f"]
Sep 06 05:07:12.191710 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:12.191688    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67b48bfc7b-gmb2f"]
Sep 06 05:07:13.153380 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:13.153349    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10a3daff-f870-489f-ab74-9d15085020e5" path="/var/lib/kubelet/pods/10a3daff-f870-489f-ab74-9d15085020e5/volumes"
Sep 06 05:07:17.180098 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:17.180067    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:30.100198 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:30.100108    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:30.199700 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:30.199667    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:31.636183 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.636148    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:31.722659 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.722622    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:31.723190 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.723169    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"]
Sep 06 05:07:31.723569 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.723556    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10a3daff-f870-489f-ab74-9d15085020e5" containerName="manager"
Sep 06 05:07:31.723611 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.723571    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="10a3daff-f870-489f-ab74-9d15085020e5" containerName="manager"
Sep 06 05:07:31.723645 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.723636    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="10a3daff-f870-489f-ab74-9d15085020e5" containerName="manager"
Sep 06 05:07:31.729764 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.729745    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:31.732475 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.732450    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 06 05:07:31.732571 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.732491    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 05:07:31.732571 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.732521    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 06 05:07:31.733368 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.733351    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-d2kzq\""
Sep 06 05:07:31.733368 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.733360    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 06 05:07:31.737724 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.737354    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"]
Sep 06 05:07:31.874923 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.874888    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:31.898214 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.898138    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d95gp\" (UniqueName: \"kubernetes.io/projected/28d975fb-da06-49c7-9a42-9a0970c403fb-kube-api-access-d95gp\") pod \"maas-api-7b97c6dbfb-p9hvv\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") " pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:31.898214 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.898177    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d975fb-da06-49c7-9a42-9a0970c403fb-openshift-service-ca\") pod \"maas-api-7b97c6dbfb-p9hvv\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") " pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:31.898214 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.898201    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28d975fb-da06-49c7-9a42-9a0970c403fb-metrics-certs\") pod \"maas-api-7b97c6dbfb-p9hvv\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") " pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:31.898472 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.898397    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28d975fb-da06-49c7-9a42-9a0970c403fb-maas-api-tls\") pod \"maas-api-7b97c6dbfb-p9hvv\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") " pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:31.974246 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.974215    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:31.999421 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.999387    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28d975fb-da06-49c7-9a42-9a0970c403fb-maas-api-tls\") pod \"maas-api-7b97c6dbfb-p9hvv\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") " pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:31.999584 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.999449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d95gp\" (UniqueName: \"kubernetes.io/projected/28d975fb-da06-49c7-9a42-9a0970c403fb-kube-api-access-d95gp\") pod \"maas-api-7b97c6dbfb-p9hvv\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") " pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:31.999584 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.999477    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d975fb-da06-49c7-9a42-9a0970c403fb-openshift-service-ca\") pod \"maas-api-7b97c6dbfb-p9hvv\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") " pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:31.999584 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:31.999499    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28d975fb-da06-49c7-9a42-9a0970c403fb-metrics-certs\") pod \"maas-api-7b97c6dbfb-p9hvv\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") " pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:32.000214 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.000186    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d975fb-da06-49c7-9a42-9a0970c403fb-openshift-service-ca\") pod \"maas-api-7b97c6dbfb-p9hvv\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") " pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:32.002007 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.001984    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28d975fb-da06-49c7-9a42-9a0970c403fb-maas-api-tls\") pod \"maas-api-7b97c6dbfb-p9hvv\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") " pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:32.002106 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.002093    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28d975fb-da06-49c7-9a42-9a0970c403fb-metrics-certs\") pod \"maas-api-7b97c6dbfb-p9hvv\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") " pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:32.015002 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.014977    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d95gp\" (UniqueName: \"kubernetes.io/projected/28d975fb-da06-49c7-9a42-9a0970c403fb-kube-api-access-d95gp\") pod \"maas-api-7b97c6dbfb-p9hvv\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") " pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:32.042063 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.042000    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:32.100010 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.099968    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:32.228943 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.228861    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:32.326079 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.326033    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"]
Sep 06 05:07:32.330240 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:07:32.330211    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28d975fb_da06_49c7_9a42_9a0970c403fb.slice/crio-8c213478f7db8eb7c5d64e07813c8918d873aef51831dec0e02ee1fd57ddf466 WatchSource:0}: Error finding container 8c213478f7db8eb7c5d64e07813c8918d873aef51831dec0e02ee1fd57ddf466: Status 404 returned error can't find the container with id 8c213478f7db8eb7c5d64e07813c8918d873aef51831dec0e02ee1fd57ddf466
Sep 06 05:07:32.431532 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.431493    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-b5x42"]
Sep 06 05:07:32.436867 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.436839    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-b5x42"
Sep 06 05:07:32.439195 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.439174    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 06 05:07:32.439324 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.439203    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2ntxn\""
Sep 06 05:07:32.445003 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.444939    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-b5x42"]
Sep 06 05:07:32.604645 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.604551    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r548g\" (UniqueName: \"kubernetes.io/projected/2c73f38f-cd1d-4b43-9f88-e7a06578fdfc-kube-api-access-r548g\") pod \"payload-processing-5b54b7fbb7-b5x42\" (UID: \"2c73f38f-cd1d-4b43-9f88-e7a06578fdfc\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-b5x42"
Sep 06 05:07:32.604796 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.604679    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c73f38f-cd1d-4b43-9f88-e7a06578fdfc-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-b5x42\" (UID: \"2c73f38f-cd1d-4b43-9f88-e7a06578fdfc\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-b5x42"
Sep 06 05:07:32.705852 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.705816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r548g\" (UniqueName: \"kubernetes.io/projected/2c73f38f-cd1d-4b43-9f88-e7a06578fdfc-kube-api-access-r548g\") pod \"payload-processing-5b54b7fbb7-b5x42\" (UID: \"2c73f38f-cd1d-4b43-9f88-e7a06578fdfc\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-b5x42"
Sep 06 05:07:32.706259 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.705933    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c73f38f-cd1d-4b43-9f88-e7a06578fdfc-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-b5x42\" (UID: \"2c73f38f-cd1d-4b43-9f88-e7a06578fdfc\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-b5x42"
Sep 06 05:07:32.706669 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.706638    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c73f38f-cd1d-4b43-9f88-e7a06578fdfc-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-b5x42\" (UID: \"2c73f38f-cd1d-4b43-9f88-e7a06578fdfc\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-b5x42"
Sep 06 05:07:32.713135 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.713114    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r548g\" (UniqueName: \"kubernetes.io/projected/2c73f38f-cd1d-4b43-9f88-e7a06578fdfc-kube-api-access-r548g\") pod \"payload-processing-5b54b7fbb7-b5x42\" (UID: \"2c73f38f-cd1d-4b43-9f88-e7a06578fdfc\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-b5x42"
Sep 06 05:07:32.751103 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:32.751074    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-b5x42"
Sep 06 05:07:33.021204 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:33.020989    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-b5x42"]
Sep 06 05:07:33.023627 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:07:33.023596    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c73f38f_cd1d_4b43_9f88_e7a06578fdfc.slice/crio-ee82faf7bed1bae2cf21a9e04ecb3e8621125aaa379b806d9fd8f02dc5a3fde8 WatchSource:0}: Error finding container ee82faf7bed1bae2cf21a9e04ecb3e8621125aaa379b806d9fd8f02dc5a3fde8: Status 404 returned error can't find the container with id ee82faf7bed1bae2cf21a9e04ecb3e8621125aaa379b806d9fd8f02dc5a3fde8
Sep 06 05:07:33.273432 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:33.273349    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv" event={"ID":"28d975fb-da06-49c7-9a42-9a0970c403fb","Type":"ContainerStarted","Data":"8c213478f7db8eb7c5d64e07813c8918d873aef51831dec0e02ee1fd57ddf466"}
Sep 06 05:07:33.274836 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:33.274800    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-b5x42" event={"ID":"2c73f38f-cd1d-4b43-9f88-e7a06578fdfc","Type":"ContainerStarted","Data":"ee82faf7bed1bae2cf21a9e04ecb3e8621125aaa379b806d9fd8f02dc5a3fde8"}
Sep 06 05:07:36.296402 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:36.296361    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv" event={"ID":"28d975fb-da06-49c7-9a42-9a0970c403fb","Type":"ContainerStarted","Data":"f0d8a3e32b383111280e5a0f9dab508ec88640b343bf7e2a58702cbb350c29bd"}
Sep 06 05:07:36.297278 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:36.297250    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:36.299332 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:36.299304    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-b5x42" event={"ID":"2c73f38f-cd1d-4b43-9f88-e7a06578fdfc","Type":"ContainerStarted","Data":"c8eae3a6473e095bdfa501419b86a79d2e4161658331a96c7589611b9515373b"}
Sep 06 05:07:36.299841 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:36.299814    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-b5x42"
Sep 06 05:07:36.312991 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:36.312933    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv" podStartSLOduration=1.8340665619999998 podStartE2EDuration="5.312913393s" totalImagesPullingTime="3.478846831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:07:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:07:36.311961465 +0000 UTC m=+723.681416728" watchObservedRunningTime="2026-09-06 05:07:36.312913393 +0000 UTC m=+723.682368642"
Sep 06 05:07:36.329913 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:36.329863    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-b5x42" podStartSLOduration=1.490290572 podStartE2EDuration="4.329847785s" totalImagesPullingTime="2.839557213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:07:36.328387727 +0000 UTC m=+723.697842975" watchObservedRunningTime="2026-09-06 05:07:36.329847785 +0000 UTC m=+723.699303030"
Sep 06 05:07:36.337272 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:36.337245    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:42.335659 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:42.335623    2578 generic.go:356] "Generic (PLEG): container finished" podID="58864663-ba1f-4c9a-bebd-0a016c980b43" containerID="8b23de40a40eab76da54ea2a93667aede6e94253d4421e1b03b40a0072657301" exitCode=0
Sep 06 05:07:42.336249 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:42.335700    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv" event={"ID":"58864663-ba1f-4c9a-bebd-0a016c980b43","Type":"ContainerDied","Data":"8b23de40a40eab76da54ea2a93667aede6e94253d4421e1b03b40a0072657301"}
Sep 06 05:07:42.336249 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:42.336201    2578 scope.go:117] "RemoveContainer" containerID="8b23de40a40eab76da54ea2a93667aede6e94253d4421e1b03b40a0072657301"
Sep 06 05:07:42.337226 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:42.337201    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:07:42.337295 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:42.337265    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:07:42.337336 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:42.337298    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:07:43.319493 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:43.319465    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:07:43.343424 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:43.343391    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv" event={"ID":"58864663-ba1f-4c9a-bebd-0a016c980b43","Type":"ContainerStarted","Data":"f6db08947f33a9c721545f1b85c2b3bf043ae6ca185ec5a5ada7295e488896f4"}
Sep 06 05:07:43.460369 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:43.460335    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:07:43.461540 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:43.461505    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-s8dcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 06 05:07:43.461681 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:43.461554    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv" podUID="58864663-ba1f-4c9a-bebd-0a016c980b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 06 05:07:44.348268 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:44.348232    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:44.448584 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:44.448554    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:07:44.460759 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:44.460724    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-s8dcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 06 05:07:44.460912 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:44.460777    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv" podUID="58864663-ba1f-4c9a-bebd-0a016c980b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 06 05:07:44.860289 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:44.860259    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:07:45.461190 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:45.461149    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-s8dcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 06 05:07:45.461563 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:45.461209    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv" podUID="58864663-ba1f-4c9a-bebd-0a016c980b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 06 05:07:46.461079 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:46.461041    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-s8dcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 06 05:07:46.461263 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:46.461100    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv" podUID="58864663-ba1f-4c9a-bebd-0a016c980b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 06 05:07:47.460358 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:47.460321    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-s8dcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 06 05:07:47.460524 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:47.460390    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv" podUID="58864663-ba1f-4c9a-bebd-0a016c980b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 06 05:07:48.315498 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:48.315469    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-b5x42"
Sep 06 05:07:48.464563 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:48.464530    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:07:48.465541 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:48.465522    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8dcv"
Sep 06 05:07:50.565088 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:07:50.565057    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:08:12.065050 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:12.064997    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:08:21.575177 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:21.575140    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:08:33.060150 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:33.060119    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:08:39.946737 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:39.946698    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:08:45.262037 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:45.261990    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:08:51.938251 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:51.938216    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"]
Sep 06 05:08:51.941723 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:51.941700    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:51.944806 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:51.944779    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 06 05:08:51.944806 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:51.944786    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 06 05:08:51.945026 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:51.944824    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 06 05:08:51.945026 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:51.944786    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9srnm\""
Sep 06 05:08:51.952136 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:51.952107    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"]
Sep 06 05:08:51.972536 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:51.972503    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:08:52.029087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.029049    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/262842d9-f0e8-4b50-ae64-00c6ac028505-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.029087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.029086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6x82\" (UniqueName: \"kubernetes.io/projected/262842d9-f0e8-4b50-ae64-00c6ac028505-kube-api-access-x6x82\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.029360 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.029115    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/262842d9-f0e8-4b50-ae64-00c6ac028505-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.029360 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.029131    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/262842d9-f0e8-4b50-ae64-00c6ac028505-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.029360 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.029228    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/262842d9-f0e8-4b50-ae64-00c6ac028505-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.029360 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.029293    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/262842d9-f0e8-4b50-ae64-00c6ac028505-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.130307 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.130269    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/262842d9-f0e8-4b50-ae64-00c6ac028505-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.130307 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.130308    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/262842d9-f0e8-4b50-ae64-00c6ac028505-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.130525 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.130347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/262842d9-f0e8-4b50-ae64-00c6ac028505-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.130525 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.130382    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/262842d9-f0e8-4b50-ae64-00c6ac028505-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.130525 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.130442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/262842d9-f0e8-4b50-ae64-00c6ac028505-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.130525 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.130460    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6x82\" (UniqueName: \"kubernetes.io/projected/262842d9-f0e8-4b50-ae64-00c6ac028505-kube-api-access-x6x82\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.130834 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.130800    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/262842d9-f0e8-4b50-ae64-00c6ac028505-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.130959 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.130882    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/262842d9-f0e8-4b50-ae64-00c6ac028505-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.130959 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.130892    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/262842d9-f0e8-4b50-ae64-00c6ac028505-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.132724 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.132697    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/262842d9-f0e8-4b50-ae64-00c6ac028505-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.132887 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.132868    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/262842d9-f0e8-4b50-ae64-00c6ac028505-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.139991 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.139970    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6x82\" (UniqueName: \"kubernetes.io/projected/262842d9-f0e8-4b50-ae64-00c6ac028505-kube-api-access-x6x82\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9dzz\" (UID: \"262842d9-f0e8-4b50-ae64-00c6ac028505\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.255000 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.254914    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:08:52.527130 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.527102    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"]
Sep 06 05:08:52.529190 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:08:52.529159    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod262842d9_f0e8_4b50_ae64_00c6ac028505.slice/crio-968a264a3e05b1f94ed56ad128ba99a9283c4e903c379f6232c93a3c667f39cb WatchSource:0}: Error finding container 968a264a3e05b1f94ed56ad128ba99a9283c4e903c379f6232c93a3c667f39cb: Status 404 returned error can't find the container with id 968a264a3e05b1f94ed56ad128ba99a9283c4e903c379f6232c93a3c667f39cb
Sep 06 05:08:52.699853 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:52.699818    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz" event={"ID":"262842d9-f0e8-4b50-ae64-00c6ac028505","Type":"ContainerStarted","Data":"968a264a3e05b1f94ed56ad128ba99a9283c4e903c379f6232c93a3c667f39cb"}
Sep 06 05:08:53.706813 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:53.706779    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz" event={"ID":"262842d9-f0e8-4b50-ae64-00c6ac028505","Type":"ContainerStarted","Data":"3329873b1cda2812d4e6f0a8bdfca066b584659d2473532b6518093d86c1bc94"}
Sep 06 05:08:53.722477 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:08:53.722427    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz" podStartSLOduration=1.76431869 podStartE2EDuration="2.72241161s" totalImagesPullingTime="958.09292ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:08:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:08:53.721553659 +0000 UTC m=+801.091008905" watchObservedRunningTime="2026-09-06 05:08:53.72241161 +0000 UTC m=+801.091866856"
Sep 06 05:09:02.255296 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:02.255247    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:09:02.255727 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:02.255328    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:09:02.267956 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:02.267924    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:09:02.765825 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:02.765797    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9dzz"
Sep 06 05:09:15.648708 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.648670    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"]
Sep 06 05:09:15.653829 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.653808    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.656960 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.656935    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 06 05:09:15.659231 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.659208    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"]
Sep 06 05:09:15.672971 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.672946    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:09:15.735073 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.735041    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69rhb\" (UniqueName: \"kubernetes.io/projected/59927d2d-edf3-473b-a818-0f80ca82ac6b-kube-api-access-69rhb\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.735225 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.735079    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59927d2d-edf3-473b-a818-0f80ca82ac6b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.735225 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.735102    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59927d2d-edf3-473b-a818-0f80ca82ac6b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.735225 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.735192    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59927d2d-edf3-473b-a818-0f80ca82ac6b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.735328 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.735226    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59927d2d-edf3-473b-a818-0f80ca82ac6b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.735328 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.735249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59927d2d-edf3-473b-a818-0f80ca82ac6b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.836280 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.836251    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59927d2d-edf3-473b-a818-0f80ca82ac6b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.836468 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.836292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59927d2d-edf3-473b-a818-0f80ca82ac6b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.836468 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.836317    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59927d2d-edf3-473b-a818-0f80ca82ac6b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.836468 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.836365    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69rhb\" (UniqueName: \"kubernetes.io/projected/59927d2d-edf3-473b-a818-0f80ca82ac6b-kube-api-access-69rhb\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.836468 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.836382    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59927d2d-edf3-473b-a818-0f80ca82ac6b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.836468 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.836405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59927d2d-edf3-473b-a818-0f80ca82ac6b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.836761 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.836739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59927d2d-edf3-473b-a818-0f80ca82ac6b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.836832 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.836768    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59927d2d-edf3-473b-a818-0f80ca82ac6b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.836832 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.836822    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59927d2d-edf3-473b-a818-0f80ca82ac6b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.838573 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.838536    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59927d2d-edf3-473b-a818-0f80ca82ac6b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.838763 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.838744    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59927d2d-edf3-473b-a818-0f80ca82ac6b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.843469 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.843448    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69rhb\" (UniqueName: \"kubernetes.io/projected/59927d2d-edf3-473b-a818-0f80ca82ac6b-kube-api-access-69rhb\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8\" (UID: \"59927d2d-edf3-473b-a818-0f80ca82ac6b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:15.963908 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:15.963817    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:16.234119 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:16.234095    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"]
Sep 06 05:09:16.236179 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:09:16.236144    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59927d2d_edf3_473b_a818_0f80ca82ac6b.slice/crio-abe58af130369cbc5c63472fc3e678507835d6d99ef29afb4536ad22de7d13d5 WatchSource:0}: Error finding container abe58af130369cbc5c63472fc3e678507835d6d99ef29afb4536ad22de7d13d5: Status 404 returned error can't find the container with id abe58af130369cbc5c63472fc3e678507835d6d99ef29afb4536ad22de7d13d5
Sep 06 05:09:16.826977 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:16.826940    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8" event={"ID":"59927d2d-edf3-473b-a818-0f80ca82ac6b","Type":"ContainerStarted","Data":"4dc176b9759f1b41d57a37e71996ca8b34eff659b614df3a4ad3bc3936a3ad01"}
Sep 06 05:09:16.826977 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:16.826984    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8" event={"ID":"59927d2d-edf3-473b-a818-0f80ca82ac6b","Type":"ContainerStarted","Data":"abe58af130369cbc5c63472fc3e678507835d6d99ef29afb4536ad22de7d13d5"}
Sep 06 05:09:16.842447 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:16.842395    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8" podStartSLOduration=1.6756963520000001 podStartE2EDuration="1.842378671s" totalImagesPullingTime="166.682319ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:09:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:09:16.841439948 +0000 UTC m=+824.210895195" watchObservedRunningTime="2026-09-06 05:09:16.842378671 +0000 UTC m=+824.211833917"
Sep 06 05:09:25.964732 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:25.964698    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:25.965120 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:25.964749    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:25.977343 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:25.977318    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:09:26.885734 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:09:26.885705    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8"
Sep 06 05:10:23.386651 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:23.386603    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c5fbddd7b-zvn8z"]
Sep 06 05:10:23.387225 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:23.386844    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c5fbddd7b-zvn8z" podUID="5ee71311-c437-4c43-b4a5-bdeb974f36bb" containerName="authorino" containerID="cri-o://21b71a5536a10a31f0b374e7ef2975c2f367cf53dae917fc5f6eecbc643cd2fd" gracePeriod=30
Sep 06 05:10:23.639921 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:23.639849    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c5fbddd7b-zvn8z"
Sep 06 05:10:23.754487 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:23.754452    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwpnm\" (UniqueName: \"kubernetes.io/projected/5ee71311-c437-4c43-b4a5-bdeb974f36bb-kube-api-access-mwpnm\") pod \"5ee71311-c437-4c43-b4a5-bdeb974f36bb\" (UID: \"5ee71311-c437-4c43-b4a5-bdeb974f36bb\") "
Sep 06 05:10:23.754662 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:23.754535    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ee71311-c437-4c43-b4a5-bdeb974f36bb-tls-cert\") pod \"5ee71311-c437-4c43-b4a5-bdeb974f36bb\" (UID: \"5ee71311-c437-4c43-b4a5-bdeb974f36bb\") "
Sep 06 05:10:23.756627 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:23.756598    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ee71311-c437-4c43-b4a5-bdeb974f36bb-kube-api-access-mwpnm" (OuterVolumeSpecName: "kube-api-access-mwpnm") pod "5ee71311-c437-4c43-b4a5-bdeb974f36bb" (UID: "5ee71311-c437-4c43-b4a5-bdeb974f36bb"). InnerVolumeSpecName "kube-api-access-mwpnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:10:23.764931 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:23.764904    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ee71311-c437-4c43-b4a5-bdeb974f36bb-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5ee71311-c437-4c43-b4a5-bdeb974f36bb" (UID: "5ee71311-c437-4c43-b4a5-bdeb974f36bb"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:10:23.856174 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:23.856132    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwpnm\" (UniqueName: \"kubernetes.io/projected/5ee71311-c437-4c43-b4a5-bdeb974f36bb-kube-api-access-mwpnm\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:10:23.856174 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:23.856166    2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ee71311-c437-4c43-b4a5-bdeb974f36bb-tls-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:10:23.870784 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:23.870744    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:24.168107 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:24.168070    2578 generic.go:356] "Generic (PLEG): container finished" podID="5ee71311-c437-4c43-b4a5-bdeb974f36bb" containerID="21b71a5536a10a31f0b374e7ef2975c2f367cf53dae917fc5f6eecbc643cd2fd" exitCode=0
Sep 06 05:10:24.168279 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:24.168118    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c5fbddd7b-zvn8z"
Sep 06 05:10:24.168279 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:24.168158    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c5fbddd7b-zvn8z" event={"ID":"5ee71311-c437-4c43-b4a5-bdeb974f36bb","Type":"ContainerDied","Data":"21b71a5536a10a31f0b374e7ef2975c2f367cf53dae917fc5f6eecbc643cd2fd"}
Sep 06 05:10:24.168279 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:24.168203    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c5fbddd7b-zvn8z" event={"ID":"5ee71311-c437-4c43-b4a5-bdeb974f36bb","Type":"ContainerDied","Data":"f5f3a6b9642e2f0f20d4fee1535532dc8de876254aabd070184aa0698b0013d8"}
Sep 06 05:10:24.168279 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:24.168220    2578 scope.go:117] "RemoveContainer" containerID="21b71a5536a10a31f0b374e7ef2975c2f367cf53dae917fc5f6eecbc643cd2fd"
Sep 06 05:10:24.177420 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:24.177390    2578 scope.go:117] "RemoveContainer" containerID="21b71a5536a10a31f0b374e7ef2975c2f367cf53dae917fc5f6eecbc643cd2fd"
Sep 06 05:10:24.177741 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:10:24.177721    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21b71a5536a10a31f0b374e7ef2975c2f367cf53dae917fc5f6eecbc643cd2fd\": container with ID starting with 21b71a5536a10a31f0b374e7ef2975c2f367cf53dae917fc5f6eecbc643cd2fd not found: ID does not exist" containerID="21b71a5536a10a31f0b374e7ef2975c2f367cf53dae917fc5f6eecbc643cd2fd"
Sep 06 05:10:24.177800 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:24.177751    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21b71a5536a10a31f0b374e7ef2975c2f367cf53dae917fc5f6eecbc643cd2fd"} err="failed to get container status \"21b71a5536a10a31f0b374e7ef2975c2f367cf53dae917fc5f6eecbc643cd2fd\": rpc error: code = NotFound desc = could not find container \"21b71a5536a10a31f0b374e7ef2975c2f367cf53dae917fc5f6eecbc643cd2fd\": container with ID starting with 21b71a5536a10a31f0b374e7ef2975c2f367cf53dae917fc5f6eecbc643cd2fd not found: ID does not exist"
Sep 06 05:10:24.189332 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:24.189298    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c5fbddd7b-zvn8z"]
Sep 06 05:10:24.203501 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:24.203465    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c5fbddd7b-zvn8z"]
Sep 06 05:10:25.153834 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:25.153800    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ee71311-c437-4c43-b4a5-bdeb974f36bb" path="/var/lib/kubelet/pods/5ee71311-c437-4c43-b4a5-bdeb974f36bb/volumes"
Sep 06 05:10:25.337622 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:25.337591    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:40.564438 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:40.564403    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:40.697214 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:40.697176    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:40.854897 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:40.854808    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"]
Sep 06 05:10:40.855423 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:40.855397    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ee71311-c437-4c43-b4a5-bdeb974f36bb" containerName="authorino"
Sep 06 05:10:40.855423 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:40.855423    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ee71311-c437-4c43-b4a5-bdeb974f36bb" containerName="authorino"
Sep 06 05:10:40.855632 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:40.855545    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ee71311-c437-4c43-b4a5-bdeb974f36bb" containerName="authorino"
Sep 06 05:10:40.860052 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:40.860029    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:40.862402 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:40.862371    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-d3a1cb-openshift-default-dockercfg-69wnk\""
Sep 06 05:10:40.870675 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:40.870645    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"]
Sep 06 05:10:40.892147 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:40.892103    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:41.015346 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.015290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-credential-socket\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.015550 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.015354    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istiod-ca-cert\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.015550 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.015470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-podinfo\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.015550 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.015497    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-token\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.015550 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.015536    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f4d9\" (UniqueName: \"kubernetes.io/projected/e2a26e4c-c0f3-4851-b505-febe3bc6806e-kube-api-access-6f4d9\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.015792 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.015622    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-workload-certs\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.015792 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.015679    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-workload-socket\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.015792 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.015723    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-envoy\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.015792 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.015754    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-data\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.084507 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.084470    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:41.116656 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.116571    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-envoy\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.116656 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.116611    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-data\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.116656 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.116652    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-credential-socket\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.116927 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.116676    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istiod-ca-cert\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.116927 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.116722    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-podinfo\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.116927 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.116745    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-token\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.116927 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.116778    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f4d9\" (UniqueName: \"kubernetes.io/projected/e2a26e4c-c0f3-4851-b505-febe3bc6806e-kube-api-access-6f4d9\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.117160 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.116950    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-workload-certs\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.117160 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.117040    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-workload-socket\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.117160 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.117133    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-credential-socket\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.117303 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.117168    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-data\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.117378 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.117357    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-workload-socket\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.117428 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.117363    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-workload-certs\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.117595 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.117571    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istiod-ca-cert\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.119141 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.119116    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-envoy\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.119400 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.119382    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-podinfo\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.125308 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.125281    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-token\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.125434 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.125334    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f4d9\" (UniqueName: \"kubernetes.io/projected/e2a26e4c-c0f3-4851-b505-febe3bc6806e-kube-api-access-6f4d9\") pod \"e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") " pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.173141 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.173109    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:41.451596 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.451574    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"]
Sep 06 05:10:41.453933 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:10:41.453900    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2a26e4c_c0f3_4851_b505_febe3bc6806e.slice/crio-fde9079706bc11dbad500612ff48c6d9006d8d1be65d6e0f12bef9a6a2f9153e WatchSource:0}: Error finding container fde9079706bc11dbad500612ff48c6d9006d8d1be65d6e0f12bef9a6a2f9153e: Status 404 returned error can't find the container with id fde9079706bc11dbad500612ff48c6d9006d8d1be65d6e0f12bef9a6a2f9153e
Sep 06 05:10:41.456176 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.456143    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:10:41.456278 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.456210    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:10:41.456278 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.456240    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:10:41.929300 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:41.929262    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:42.025499 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:42.025465    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:42.260905 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:42.260824    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47" event={"ID":"e2a26e4c-c0f3-4851-b505-febe3bc6806e","Type":"ContainerStarted","Data":"48006ff27cc35c9a0f83dea008982324a19401fdc58da60a58ffdf4aab7aa4db"}
Sep 06 05:10:42.260905 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:42.260875    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47" event={"ID":"e2a26e4c-c0f3-4851-b505-febe3bc6806e","Type":"ContainerStarted","Data":"fde9079706bc11dbad500612ff48c6d9006d8d1be65d6e0f12bef9a6a2f9153e"}
Sep 06 05:10:42.280978 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:42.280930    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47" podStartSLOduration=2.28091394 podStartE2EDuration="2.28091394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:10:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:10:42.279635595 +0000 UTC m=+909.649090867" watchObservedRunningTime="2026-09-06 05:10:42.28091394 +0000 UTC m=+909.650369187"
Sep 06 05:10:42.367408 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:42.367380    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:43.173719 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:43.173687    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:43.178753 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:43.178728    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:43.265512 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:43.265478    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:43.266433 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:43.266413    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:10:44.166847 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:44.166818    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:44.271395 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:44.271356    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:45.273670 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.273638    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"]
Sep 06 05:10:45.277373 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.277353    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:45.279362 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.279340    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-d3a1cb-serving-cert\""
Sep 06 05:10:45.286778 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.286754    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"]
Sep 06 05:10:45.344493 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.344460    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:45.455212 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.455174    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:45.455405 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.455220    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-maas-api-tls\") pod \"maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:45.455405 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.455255    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljq9g\" (UniqueName: \"kubernetes.io/projected/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-kube-api-access-ljq9g\") pod \"maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:45.455548 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.455431    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-metrics-certs\") pod \"maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:45.466913 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.466881    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:45.556251 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.556150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljq9g\" (UniqueName: \"kubernetes.io/projected/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-kube-api-access-ljq9g\") pod \"maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:45.556432 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.556277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-metrics-certs\") pod \"maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:45.556432 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.556335    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:45.556432 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.556372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-maas-api-tls\") pod \"maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:45.556609 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:10:45.556491    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-serving-cert: secret "maas-api-e2e-inf-a-d3a1cb-serving-cert" not found
Sep 06 05:10:45.556609 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:10:45.556573    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-maas-api-tls podName:e5493df2-7b81-4bab-a62b-9b3c9bfd0d36 nodeName:}" failed. No retries permitted until 2026-09-06 05:10:46.05655081 +0000 UTC m=+913.426006035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-maas-api-tls") pod "maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr" (UID: "e5493df2-7b81-4bab-a62b-9b3c9bfd0d36") : secret "maas-api-e2e-inf-a-d3a1cb-serving-cert" not found
Sep 06 05:10:45.557139 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.557109    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:45.559372 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.559353    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-metrics-certs\") pod \"maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:45.564772 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.564741    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljq9g\" (UniqueName: \"kubernetes.io/projected/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-kube-api-access-ljq9g\") pod \"maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:45.639734 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.639702    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:45.890720 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.890679    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:45.987637 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:45.987601    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:46.061000 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:46.060965    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-maas-api-tls\") pod \"maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:46.063445 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:46.063419    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-maas-api-tls\") pod \"maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:46.189441 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:46.189348    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:46.461786 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:46.461765    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"]
Sep 06 05:10:46.463855 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:10:46.463829    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5493df2_7b81_4bab_a62b_9b3c9bfd0d36.slice/crio-2ccd6f646a36f3b14e22dd605f6d3c68017915028976073627dfb503e0d3ed54 WatchSource:0}: Error finding container 2ccd6f646a36f3b14e22dd605f6d3c68017915028976073627dfb503e0d3ed54: Status 404 returned error can't find the container with id 2ccd6f646a36f3b14e22dd605f6d3c68017915028976073627dfb503e0d3ed54
Sep 06 05:10:47.287683 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:47.287650    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr" event={"ID":"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36","Type":"ContainerStarted","Data":"2ccd6f646a36f3b14e22dd605f6d3c68017915028976073627dfb503e0d3ed54"}
Sep 06 05:10:47.887450 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:47.887399    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:47.988331 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:47.988285    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:48.104752 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:48.104703    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:48.295826 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:48.295739    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr" event={"ID":"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36","Type":"ContainerStarted","Data":"621a572ec0851e48a669583d7c031bd01e81eaa81957960e9ea56585a2cb90ca"}
Sep 06 05:10:48.295985 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:48.295837    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:48.315287 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:48.315230    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr" podStartSLOduration=2.5325300139999998 podStartE2EDuration="3.3152133s" totalImagesPullingTime="782.683286ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:10:48.314360295 +0000 UTC m=+915.683815541" watchObservedRunningTime="2026-09-06 05:10:48.3152133 +0000 UTC m=+915.684668545"
Sep 06 05:10:48.398807 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:48.398778    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:48.631446 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:48.631393    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:49.036913 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:49.036834    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:50.138364 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.138331    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:50.407272 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.407186    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"]
Sep 06 05:10:50.410935 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.410913    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.413066 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.413043    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-9afd7f-openshift-default-dockercfg-79r7h\""
Sep 06 05:10:50.421068 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.421043    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"]
Sep 06 05:10:50.439699 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.439662    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:50.508362 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.508326    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-credential-socket\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.508622 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.508596    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b750788-2244-4402-8fc0-f52877293d1b-istio-podinfo\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.508728 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.508680    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-workload-socket\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.508784 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.508727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-istio-envoy\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.508835 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.508790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-istio-data\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.508878 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.508844    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-workload-certs\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.508878 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.508875    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b750788-2244-4402-8fc0-f52877293d1b-istio-token\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.508959 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.508894    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b750788-2244-4402-8fc0-f52877293d1b-istiod-ca-cert\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.508959 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.508946    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfmh2\" (UniqueName: \"kubernetes.io/projected/3b750788-2244-4402-8fc0-f52877293d1b-kube-api-access-cfmh2\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.535388 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.535359    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:50.610281 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.610242    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfmh2\" (UniqueName: \"kubernetes.io/projected/3b750788-2244-4402-8fc0-f52877293d1b-kube-api-access-cfmh2\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.610460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.610299    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-credential-socket\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.610460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.610328    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b750788-2244-4402-8fc0-f52877293d1b-istio-podinfo\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.610460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.610438    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-workload-socket\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.610719 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.610478    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-istio-envoy\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.610719 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.610514    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-istio-data\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.610719 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.610568    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-workload-certs\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.610719 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.610601    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b750788-2244-4402-8fc0-f52877293d1b-istio-token\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.610719 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.610633    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b750788-2244-4402-8fc0-f52877293d1b-istiod-ca-cert\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.610989 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.610767    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-credential-socket\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.611231 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.611207    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-workload-socket\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.611390 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.611218    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-istio-data\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.611521 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.611409    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-workload-certs\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.611521 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.611430    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b750788-2244-4402-8fc0-f52877293d1b-istiod-ca-cert\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.613390 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.613358    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-istio-envoy\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.613903 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.613865    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b750788-2244-4402-8fc0-f52877293d1b-istio-podinfo\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.618912 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.618887    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfmh2\" (UniqueName: \"kubernetes.io/projected/3b750788-2244-4402-8fc0-f52877293d1b-kube-api-access-cfmh2\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.619090 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.619055    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b750788-2244-4402-8fc0-f52877293d1b-istio-token\") pod \"e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") " pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.635419 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.635392    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:50.722532 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.722450    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:50.994466 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.994433    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"]
Sep 06 05:10:50.996143 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:10:50.996118    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b750788_2244_4402_8fc0_f52877293d1b.slice/crio-0bb5148e01ad2249c659e7424c1eeec2df3f38cef2357ed6895175b1b17c82e3 WatchSource:0}: Error finding container 0bb5148e01ad2249c659e7424c1eeec2df3f38cef2357ed6895175b1b17c82e3: Status 404 returned error can't find the container with id 0bb5148e01ad2249c659e7424c1eeec2df3f38cef2357ed6895175b1b17c82e3
Sep 06 05:10:50.998487 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.998451    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:10:50.998573 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.998518    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:10:50.998573 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:50.998546    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:10:51.319035 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.318927    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9" event={"ID":"3b750788-2244-4402-8fc0-f52877293d1b","Type":"ContainerStarted","Data":"6fe75eb19824bc76437613cd82384375a16e992bff585a7eccc09bebceb5700e"}
Sep 06 05:10:51.319384 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.319274    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9" event={"ID":"3b750788-2244-4402-8fc0-f52877293d1b","Type":"ContainerStarted","Data":"0bb5148e01ad2249c659e7424c1eeec2df3f38cef2357ed6895175b1b17c82e3"}
Sep 06 05:10:51.338167 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.338118    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9" podStartSLOduration=1.338103244 podStartE2EDuration="1.338103244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:10:51.336846757 +0000 UTC m=+918.706302028" watchObservedRunningTime="2026-09-06 05:10:51.338103244 +0000 UTC m=+918.707558490"
Sep 06 05:10:51.573290 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.573210    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"]
Sep 06 05:10:51.576839 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.576818    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:51.578913 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.578892    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-64514da2-serving-cert\""
Sep 06 05:10:51.585073 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.585046    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"]
Sep 06 05:10:51.605537 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.605499    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:51.621451 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.621421    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9259b7e0-fe7b-462e-bc7b-f651b027fc44-openshift-service-ca\") pod \"maas-api-e2e-ait-64514da2-6fc595f696-cr6wz\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:51.621571 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.621463    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gj9b\" (UniqueName: \"kubernetes.io/projected/9259b7e0-fe7b-462e-bc7b-f651b027fc44-kube-api-access-8gj9b\") pod \"maas-api-e2e-ait-64514da2-6fc595f696-cr6wz\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:51.621571 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.621553    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9259b7e0-fe7b-462e-bc7b-f651b027fc44-maas-api-tls\") pod \"maas-api-e2e-ait-64514da2-6fc595f696-cr6wz\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:51.621660 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.621583    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9259b7e0-fe7b-462e-bc7b-f651b027fc44-metrics-certs\") pod \"maas-api-e2e-ait-64514da2-6fc595f696-cr6wz\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:51.707697 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.707658    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:51.722836 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.722804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9259b7e0-fe7b-462e-bc7b-f651b027fc44-openshift-service-ca\") pod \"maas-api-e2e-ait-64514da2-6fc595f696-cr6wz\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:51.722836 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.722839    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gj9b\" (UniqueName: \"kubernetes.io/projected/9259b7e0-fe7b-462e-bc7b-f651b027fc44-kube-api-access-8gj9b\") pod \"maas-api-e2e-ait-64514da2-6fc595f696-cr6wz\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:51.723073 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.722879    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9259b7e0-fe7b-462e-bc7b-f651b027fc44-maas-api-tls\") pod \"maas-api-e2e-ait-64514da2-6fc595f696-cr6wz\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:51.723073 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.722905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9259b7e0-fe7b-462e-bc7b-f651b027fc44-metrics-certs\") pod \"maas-api-e2e-ait-64514da2-6fc595f696-cr6wz\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:51.723073 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.722954    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:51.723633 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.723602    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9259b7e0-fe7b-462e-bc7b-f651b027fc44-openshift-service-ca\") pod \"maas-api-e2e-ait-64514da2-6fc595f696-cr6wz\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:51.725425 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.725405    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9259b7e0-fe7b-462e-bc7b-f651b027fc44-metrics-certs\") pod \"maas-api-e2e-ait-64514da2-6fc595f696-cr6wz\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:51.725512 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.725467    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9259b7e0-fe7b-462e-bc7b-f651b027fc44-maas-api-tls\") pod \"maas-api-e2e-ait-64514da2-6fc595f696-cr6wz\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:51.730642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.730620    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gj9b\" (UniqueName: \"kubernetes.io/projected/9259b7e0-fe7b-462e-bc7b-f651b027fc44-kube-api-access-8gj9b\") pod \"maas-api-e2e-ait-64514da2-6fc595f696-cr6wz\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:51.807473 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.807435    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:51.893695 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:51.893657    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:52.174544 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.174394    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"]
Sep 06 05:10:52.177362 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:10:52.177334    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9259b7e0_fe7b_462e_bc7b_f651b027fc44.slice/crio-98e7560c214ab7aa94f0a18f7e7829072c3fc660f22df1876cb32e632f0194a1 WatchSource:0}: Error finding container 98e7560c214ab7aa94f0a18f7e7829072c3fc660f22df1876cb32e632f0194a1: Status 404 returned error can't find the container with id 98e7560c214ab7aa94f0a18f7e7829072c3fc660f22df1876cb32e632f0194a1
Sep 06 05:10:52.204492 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.204464    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:52.325366 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.325326    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz" event={"ID":"9259b7e0-fe7b-462e-bc7b-f651b027fc44","Type":"ContainerStarted","Data":"98e7560c214ab7aa94f0a18f7e7829072c3fc660f22df1876cb32e632f0194a1"}
Sep 06 05:10:52.332828 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.332804    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:52.514760 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.514709    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"]
Sep 06 05:10:52.519944 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.519923    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"
Sep 06 05:10:52.522283 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.522262    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-64514da2-dockercfg-6llsn\""
Sep 06 05:10:52.522400 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.522281    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-64514da2\""
Sep 06 05:10:52.528418 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.528394    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"]
Sep 06 05:10:52.632500 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.632470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g59vb\" (UniqueName: \"kubernetes.io/projected/24b12420-bd61-4b5d-ae49-862aae2470d9-kube-api-access-g59vb\") pod \"payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn\" (UID: \"24b12420-bd61-4b5d-ae49-862aae2470d9\") " pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"
Sep 06 05:10:52.632673 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.632517    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b12420-bd61-4b5d-ae49-862aae2470d9-plugins-config-volume\") pod \"payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn\" (UID: \"24b12420-bd61-4b5d-ae49-862aae2470d9\") " pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"
Sep 06 05:10:52.728648 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.728611    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:52.733707 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.733668    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g59vb\" (UniqueName: \"kubernetes.io/projected/24b12420-bd61-4b5d-ae49-862aae2470d9-kube-api-access-g59vb\") pod \"payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn\" (UID: \"24b12420-bd61-4b5d-ae49-862aae2470d9\") " pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"
Sep 06 05:10:52.733707 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.733708    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b12420-bd61-4b5d-ae49-862aae2470d9-plugins-config-volume\") pod \"payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn\" (UID: \"24b12420-bd61-4b5d-ae49-862aae2470d9\") " pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"
Sep 06 05:10:52.734455 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.734430    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b12420-bd61-4b5d-ae49-862aae2470d9-plugins-config-volume\") pod \"payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn\" (UID: \"24b12420-bd61-4b5d-ae49-862aae2470d9\") " pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"
Sep 06 05:10:52.743255 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.743183    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g59vb\" (UniqueName: \"kubernetes.io/projected/24b12420-bd61-4b5d-ae49-862aae2470d9-kube-api-access-g59vb\") pod \"payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn\" (UID: \"24b12420-bd61-4b5d-ae49-862aae2470d9\") " pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"
Sep 06 05:10:52.843767 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:52.843729    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"
Sep 06 05:10:53.121942 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:53.121913    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"]
Sep 06 05:10:53.124572 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:10:53.124533    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b12420_bd61_4b5d_ae49_862aae2470d9.slice/crio-136a4b55a3432860ec4b305f6b59603baed0b7bdcd2eaa73bcfa42154ec504b6 WatchSource:0}: Error finding container 136a4b55a3432860ec4b305f6b59603baed0b7bdcd2eaa73bcfa42154ec504b6: Status 404 returned error can't find the container with id 136a4b55a3432860ec4b305f6b59603baed0b7bdcd2eaa73bcfa42154ec504b6
Sep 06 05:10:53.310819 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:53.310786    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:53.342425 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:53.342391    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz" event={"ID":"9259b7e0-fe7b-462e-bc7b-f651b027fc44","Type":"ContainerStarted","Data":"ed0ec08909accf3acd03d760fd570eae6742d6917323d8eda0d7a1eeb0b4a35a"}
Sep 06 05:10:53.343298 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:53.343251    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:53.346320 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:53.346225    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn" event={"ID":"24b12420-bd61-4b5d-ae49-862aae2470d9","Type":"ContainerStarted","Data":"7e69b4800ee0571036bc9f25e53a9cfc5823247ef50041b83d031c734cffe4e9"}
Sep 06 05:10:53.346320 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:53.346277    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn" event={"ID":"24b12420-bd61-4b5d-ae49-862aae2470d9","Type":"ContainerStarted","Data":"136a4b55a3432860ec4b305f6b59603baed0b7bdcd2eaa73bcfa42154ec504b6"}
Sep 06 05:10:53.347387 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:53.347218    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:53.347387 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:53.347287    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"
Sep 06 05:10:53.348169 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:53.348122    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:10:53.360551 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:53.360496    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz" podStartSLOduration=2.051626108 podStartE2EDuration="2.360478614s" totalImagesPullingTime="308.852506ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:10:53.358789921 +0000 UTC m=+920.728245172" watchObservedRunningTime="2026-09-06 05:10:53.360478614 +0000 UTC m=+920.729933862"
Sep 06 05:10:53.376672 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:53.376599    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn" podStartSLOduration=1.376586442 podStartE2EDuration="1.376586442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:10:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:10:53.374243345 +0000 UTC m=+920.743698591" watchObservedRunningTime="2026-09-06 05:10:53.376586442 +0000 UTC m=+920.746041686"
Sep 06 05:10:53.378420 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:53.378397    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:54.311480 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:54.311452    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:10:54.503128 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:54.503077    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:54.602684 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:54.602574    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:54.727984 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:54.727950    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:55.089556 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:55.089493    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:55.192294 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:55.192254    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:59.358735 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:59.358700    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:10:59.436233 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:59.436202    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:59.596620 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:59.596575    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:59.684869 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:59.684837    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:10:59.785210 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:10:59.785177    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:02.410975 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:02.410941    2578 generic.go:356] "Generic (PLEG): container finished" podID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" containerID="48006ff27cc35c9a0f83dea008982324a19401fdc58da60a58ffdf4aab7aa4db" exitCode=0
Sep 06 05:11:02.411476 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:02.410985    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47" event={"ID":"e2a26e4c-c0f3-4851-b505-febe3bc6806e","Type":"ContainerDied","Data":"48006ff27cc35c9a0f83dea008982324a19401fdc58da60a58ffdf4aab7aa4db"}
Sep 06 05:11:02.411476 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:02.411407    2578 scope.go:117] "RemoveContainer" containerID="48006ff27cc35c9a0f83dea008982324a19401fdc58da60a58ffdf4aab7aa4db"
Sep 06 05:11:02.412490 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:02.412460    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:02.412547 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:02.412531    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:02.412583 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:02.412565    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:02.591983 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:02.591948    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:03.421676 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:03.421639    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47" event={"ID":"e2a26e4c-c0f3-4851-b505-febe3bc6806e","Type":"ContainerStarted","Data":"03714524cf6edc2601b41fef2ea4ac530d87e5b6ff551b67184a910a8995785d"}
Sep 06 05:11:04.173817 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:04.173786    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:11:04.174087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:04.173973    2578 patch_prober.go:28] interesting pod/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 06 05:11:04.174087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:04.174043    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47" podUID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 06 05:11:04.249196 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:04.249164    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:05.174518 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:05.174486    2578 patch_prober.go:28] interesting pod/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 06 05:11:05.174963 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:05.174543    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47" podUID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 06 05:11:05.361641 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:05.361593    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"
Sep 06 05:11:06.173977 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:06.173934    2578 patch_prober.go:28] interesting pod/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 06 05:11:06.174281 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:06.174249    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47" podUID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 06 05:11:06.317947 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:06.317890    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:07.173751 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:07.173720    2578 patch_prober.go:28] interesting pod/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 06 05:11:07.173993 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:07.173964    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47" podUID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 06 05:11:08.173794 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:08.173763    2578 patch_prober.go:28] interesting pod/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 06 05:11:08.174274 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:08.174247    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47" podUID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 06 05:11:09.092434 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.092397    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:09.177954 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.177923    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:11:09.178337 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.178254    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:11:09.178912 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.178888    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:11:09.408891 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.408858    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:09.674393 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.674302    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"]
Sep 06 05:11:09.677777 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.677757    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.680001 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.679977    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-9035bc-openshift-default-dockercfg-2njhm\""
Sep 06 05:11:09.687977 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.687912    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"]
Sep 06 05:11:09.724423 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.724385    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:09.827636 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.827601    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-workload-certs\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.827836 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.827726    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-workload-socket\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.827836 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.827776    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgj9x\" (UniqueName: \"kubernetes.io/projected/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-kube-api-access-cgj9x\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.827836 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.827810    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-token\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.828009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.827835    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-data\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.828009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.827931    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-envoy\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.828009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.828002    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-podinfo\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.828201 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.828064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-credential-socket\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.828201 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.828132    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istiod-ca-cert\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.929603 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.929500    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-workload-socket\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.929603 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.929556    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgj9x\" (UniqueName: \"kubernetes.io/projected/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-kube-api-access-cgj9x\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.929603 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.929590    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-token\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.929895 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.929618    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-data\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.929895 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.929696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-envoy\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.929895 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.929756    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-podinfo\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.929895 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.929784    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-credential-socket\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.929895 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.929844    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istiod-ca-cert\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.929895 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.929889    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-workload-certs\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.930245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.929993    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-workload-socket\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.930245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.930047    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-data\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.930245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.930183    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-credential-socket\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.930366 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.930248    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-workload-certs\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.930571 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.930549    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istiod-ca-cert\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.931989 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.931970    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-envoy\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.932341 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.932323    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-podinfo\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.936675 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.936652    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-token\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.936809 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.936792    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgj9x\" (UniqueName: \"kubernetes.io/projected/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-kube-api-access-cgj9x\") pod \"e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") " pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:09.994772 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:09.994735    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:10.032492 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:10.032452    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:10.292160 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:10.292068    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"]
Sep 06 05:11:10.297995 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:11:10.297948    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e6a0a4d_5526_4e50_8fee_49ef9d2d1312.slice/crio-2821b07898ce1c95166d30cecb754ed9f9653bce1c59b16913959964473582a5 WatchSource:0}: Error finding container 2821b07898ce1c95166d30cecb754ed9f9653bce1c59b16913959964473582a5: Status 404 returned error can't find the container with id 2821b07898ce1c95166d30cecb754ed9f9653bce1c59b16913959964473582a5
Sep 06 05:11:10.302407 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:10.302353    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:10.302515 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:10.302461    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:10.302577 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:10.302530    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:10.470777 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:10.470733    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm" event={"ID":"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312","Type":"ContainerStarted","Data":"ab710e099ae8b8a7a8bfd39304f475903b9879839bf0846073d92e6f048e6727"}
Sep 06 05:11:10.470957 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:10.470806    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm" event={"ID":"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312","Type":"ContainerStarted","Data":"2821b07898ce1c95166d30cecb754ed9f9653bce1c59b16913959964473582a5"}
Sep 06 05:11:10.493079 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:10.492996    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm" podStartSLOduration=1.4929647799999999 podStartE2EDuration="1.49296478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:11:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:11:10.488588632 +0000 UTC m=+937.858043906" watchObservedRunningTime="2026-09-06 05:11:10.49296478 +0000 UTC m=+937.862420027"
Sep 06 05:11:10.995053 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:10.994996    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:12.001048 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:12.000997    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:12.377811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:12.377771    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:12.470577 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:12.470539    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:12.482025 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:12.481993    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:12.483042 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:12.482993    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:12.774418 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:12.774271    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:13.973991 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:13.973919    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:14.028718 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:14.028685    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:14.318696 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:14.318664    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:14.432544 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:14.432514    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:14.498211 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:14.498179    2578 generic.go:356] "Generic (PLEG): container finished" podID="3b750788-2244-4402-8fc0-f52877293d1b" containerID="6fe75eb19824bc76437613cd82384375a16e992bff585a7eccc09bebceb5700e" exitCode=0
Sep 06 05:11:14.498379 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:14.498256    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9" event={"ID":"3b750788-2244-4402-8fc0-f52877293d1b","Type":"ContainerDied","Data":"6fe75eb19824bc76437613cd82384375a16e992bff585a7eccc09bebceb5700e"}
Sep 06 05:11:14.498845 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:14.498828    2578 scope.go:117] "RemoveContainer" containerID="6fe75eb19824bc76437613cd82384375a16e992bff585a7eccc09bebceb5700e"
Sep 06 05:11:14.499943 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:14.499911    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:14.500063 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:14.499984    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:14.500132 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:14.500050    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:14.522399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:14.522363    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:14.803829 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:14.803796    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:15.509976 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:15.509939    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9" event={"ID":"3b750788-2244-4402-8fc0-f52877293d1b","Type":"ContainerStarted","Data":"ae970ccfd5072ade9f76bd1474bd69a5139a7e225178e246c6ac147f05b172b6"}
Sep 06 05:11:15.578217 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:15.578183    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:15.723563 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:15.723520    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:11:15.723799 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:15.723765    2578 patch_prober.go:28] interesting pod/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 06 05:11:15.723859 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:15.723838    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9" podUID="3b750788-2244-4402-8fc0-f52877293d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 06 05:11:16.480193 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:16.480154    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:16.723157 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:16.723115    2578 patch_prober.go:28] interesting pod/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 06 05:11:16.723735 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:16.723702    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9" podUID="3b750788-2244-4402-8fc0-f52877293d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 06 05:11:17.204881 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:17.204807    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:17.723265 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:17.723229    2578 patch_prober.go:28] interesting pod/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 06 05:11:17.723675 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:17.723298    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9" podUID="3b750788-2244-4402-8fc0-f52877293d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 06 05:11:17.874191 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:17.874140    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:18.126353 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:18.126315    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:18.722753 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:18.722720    2578 patch_prober.go:28] interesting pod/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 06 05:11:18.722927 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:18.722783    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9" podUID="3b750788-2244-4402-8fc0-f52877293d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 06 05:11:19.723531 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:19.723493    2578 patch_prober.go:28] interesting pod/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 06 05:11:19.723921 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:19.723554    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9" podUID="3b750788-2244-4402-8fc0-f52877293d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 06 05:11:20.302351 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:20.302316    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:20.400984 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:20.400950    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:20.523759 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:20.523728    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:20.848425 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:20.848392    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:11:20.848946 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:20.848922    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:11:20.850241 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:20.850220    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:11:26.290495 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:26.290459    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:26.400126 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:26.400088    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:26.538047 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:26.537997    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:26.728057 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:26.728002    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:31.803878 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:31.803787    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:32.365722 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.365689    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"]
Sep 06 05:11:32.366181 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.366137    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz" podUID="9259b7e0-fe7b-462e-bc7b-f651b027fc44" containerName="maas-api" containerID="cri-o://ed0ec08909accf3acd03d760fd570eae6742d6917323d8eda0d7a1eeb0b4a35a" gracePeriod=30
Sep 06 05:11:32.398085 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.397949    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:32.430959 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.430923    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"]
Sep 06 05:11:32.431246 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.431220    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn" podUID="24b12420-bd61-4b5d-ae49-862aae2470d9" containerName="payload-processing" containerID="cri-o://7e69b4800ee0571036bc9f25e53a9cfc5823247ef50041b83d031c734cffe4e9" gracePeriod=30
Sep 06 05:11:32.502099 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.501121    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:32.594814 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.594770    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:32.620686 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.620639    2578 generic.go:356] "Generic (PLEG): container finished" podID="24b12420-bd61-4b5d-ae49-862aae2470d9" containerID="7e69b4800ee0571036bc9f25e53a9cfc5823247ef50041b83d031c734cffe4e9" exitCode=0
Sep 06 05:11:32.620881 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.620765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn" event={"ID":"24b12420-bd61-4b5d-ae49-862aae2470d9","Type":"ContainerDied","Data":"7e69b4800ee0571036bc9f25e53a9cfc5823247ef50041b83d031c734cffe4e9"}
Sep 06 05:11:32.622998 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.622849    2578 generic.go:356] "Generic (PLEG): container finished" podID="9259b7e0-fe7b-462e-bc7b-f651b027fc44" containerID="ed0ec08909accf3acd03d760fd570eae6742d6917323d8eda0d7a1eeb0b4a35a" exitCode=0
Sep 06 05:11:32.622998 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.622930    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz" event={"ID":"9259b7e0-fe7b-462e-bc7b-f651b027fc44","Type":"ContainerDied","Data":"ed0ec08909accf3acd03d760fd570eae6742d6917323d8eda0d7a1eeb0b4a35a"}
Sep 06 05:11:32.686377 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.686351    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:11:32.695232 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.695152    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:32.734475 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.734433    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"
Sep 06 05:11:32.756724 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.756690    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:32.788339 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.788303    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gj9b\" (UniqueName: \"kubernetes.io/projected/9259b7e0-fe7b-462e-bc7b-f651b027fc44-kube-api-access-8gj9b\") pod \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") "
Sep 06 05:11:32.788339 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.788343    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9259b7e0-fe7b-462e-bc7b-f651b027fc44-openshift-service-ca\") pod \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") "
Sep 06 05:11:32.788725 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.788373    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9259b7e0-fe7b-462e-bc7b-f651b027fc44-metrics-certs\") pod \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") "
Sep 06 05:11:32.789114 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.789076    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9259b7e0-fe7b-462e-bc7b-f651b027fc44-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9259b7e0-fe7b-462e-bc7b-f651b027fc44" (UID: "9259b7e0-fe7b-462e-bc7b-f651b027fc44"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:11:32.789222 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.789139    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9259b7e0-fe7b-462e-bc7b-f651b027fc44-maas-api-tls\") pod \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\" (UID: \"9259b7e0-fe7b-462e-bc7b-f651b027fc44\") "
Sep 06 05:11:32.789795 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.789557    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9259b7e0-fe7b-462e-bc7b-f651b027fc44-openshift-service-ca\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:11:32.791244 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.791192    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9259b7e0-fe7b-462e-bc7b-f651b027fc44-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9259b7e0-fe7b-462e-bc7b-f651b027fc44" (UID: "9259b7e0-fe7b-462e-bc7b-f651b027fc44"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:11:32.791825 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.791793    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9259b7e0-fe7b-462e-bc7b-f651b027fc44-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9259b7e0-fe7b-462e-bc7b-f651b027fc44" (UID: "9259b7e0-fe7b-462e-bc7b-f651b027fc44"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:11:32.792079 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.792057    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9259b7e0-fe7b-462e-bc7b-f651b027fc44-kube-api-access-8gj9b" (OuterVolumeSpecName: "kube-api-access-8gj9b") pod "9259b7e0-fe7b-462e-bc7b-f651b027fc44" (UID: "9259b7e0-fe7b-462e-bc7b-f651b027fc44"). InnerVolumeSpecName "kube-api-access-8gj9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:11:32.890669 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.890581    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b12420-bd61-4b5d-ae49-862aae2470d9-plugins-config-volume\") pod \"24b12420-bd61-4b5d-ae49-862aae2470d9\" (UID: \"24b12420-bd61-4b5d-ae49-862aae2470d9\") "
Sep 06 05:11:32.890669 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.890654    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g59vb\" (UniqueName: \"kubernetes.io/projected/24b12420-bd61-4b5d-ae49-862aae2470d9-kube-api-access-g59vb\") pod \"24b12420-bd61-4b5d-ae49-862aae2470d9\" (UID: \"24b12420-bd61-4b5d-ae49-862aae2470d9\") "
Sep 06 05:11:32.891161 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.890938    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gj9b\" (UniqueName: \"kubernetes.io/projected/9259b7e0-fe7b-462e-bc7b-f651b027fc44-kube-api-access-8gj9b\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:11:32.891161 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.890958    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9259b7e0-fe7b-462e-bc7b-f651b027fc44-metrics-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:11:32.891161 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.890971    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9259b7e0-fe7b-462e-bc7b-f651b027fc44-maas-api-tls\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:11:32.891161 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.891011    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24b12420-bd61-4b5d-ae49-862aae2470d9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "24b12420-bd61-4b5d-ae49-862aae2470d9" (UID: "24b12420-bd61-4b5d-ae49-862aae2470d9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:11:32.892891 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.892859    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24b12420-bd61-4b5d-ae49-862aae2470d9-kube-api-access-g59vb" (OuterVolumeSpecName: "kube-api-access-g59vb") pod "24b12420-bd61-4b5d-ae49-862aae2470d9" (UID: "24b12420-bd61-4b5d-ae49-862aae2470d9"). InnerVolumeSpecName "kube-api-access-g59vb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:11:32.923789 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.923752    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:32.981547 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.981496    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:32.991942 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.991906    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b12420-bd61-4b5d-ae49-862aae2470d9-plugins-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:11:32.991942 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:32.991942    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g59vb\" (UniqueName: \"kubernetes.io/projected/24b12420-bd61-4b5d-ae49-862aae2470d9-kube-api-access-g59vb\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:11:33.218189 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.218150    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:33.317440 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.317405    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:33.373834 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.373802    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:33.631127 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.631061    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn" event={"ID":"24b12420-bd61-4b5d-ae49-862aae2470d9","Type":"ContainerDied","Data":"136a4b55a3432860ec4b305f6b59603baed0b7bdcd2eaa73bcfa42154ec504b6"}
Sep 06 05:11:33.631127 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.631101    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"
Sep 06 05:11:33.631895 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.631143    2578 scope.go:117] "RemoveContainer" containerID="7e69b4800ee0571036bc9f25e53a9cfc5823247ef50041b83d031c734cffe4e9"
Sep 06 05:11:33.633441 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.633412    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz" event={"ID":"9259b7e0-fe7b-462e-bc7b-f651b027fc44","Type":"ContainerDied","Data":"98e7560c214ab7aa94f0a18f7e7829072c3fc660f22df1876cb32e632f0194a1"}
Sep 06 05:11:33.633536 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.633528    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"
Sep 06 05:11:33.644215 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.643230    2578 scope.go:117] "RemoveContainer" containerID="ed0ec08909accf3acd03d760fd570eae6742d6917323d8eda0d7a1eeb0b4a35a"
Sep 06 05:11:33.654905 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.654846    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"]
Sep 06 05:11:33.655873 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.655613    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9259b7e0-fe7b-462e-bc7b-f651b027fc44" containerName="maas-api"
Sep 06 05:11:33.655873 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.655632    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9259b7e0-fe7b-462e-bc7b-f651b027fc44" containerName="maas-api"
Sep 06 05:11:33.655873 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.655650    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24b12420-bd61-4b5d-ae49-862aae2470d9" containerName="payload-processing"
Sep 06 05:11:33.655873 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.655659    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="24b12420-bd61-4b5d-ae49-862aae2470d9" containerName="payload-processing"
Sep 06 05:11:33.655873 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.655793    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9259b7e0-fe7b-462e-bc7b-f651b027fc44" containerName="maas-api"
Sep 06 05:11:33.655873 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.655809    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="24b12420-bd61-4b5d-ae49-862aae2470d9" containerName="payload-processing"
Sep 06 05:11:33.658895 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.658872    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.659073 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.659048    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"]
Sep 06 05:11:33.661508 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.661486    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-85214949-gw-openshift-default-dockercfg-nvddl\""
Sep 06 05:11:33.662674 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.662648    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64514da2-6fc595f696-cr6wz"]
Sep 06 05:11:33.672478 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.672454    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"]
Sep 06 05:11:33.680858 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.680826    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"]
Sep 06 05:11:33.682819 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.682794    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:33.686876 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.686841    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64514da2-6d95f88489-mv2kn"]
Sep 06 05:11:33.779666 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.779629    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:33.800914 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.800871    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-envoy\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.800914 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.800917    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-podinfo\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.801156 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.800955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-data\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.801156 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.801118    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-credential-socket\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.801251 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.801171    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-workload-certs\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.801251 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.801205    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3b144d5-b456-418d-948d-1b67a9cbd68e-istiod-ca-cert\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.801347 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.801263    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-token\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.801347 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.801309    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktfw5\" (UniqueName: \"kubernetes.io/projected/d3b144d5-b456-418d-948d-1b67a9cbd68e-kube-api-access-ktfw5\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.801449 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.801375    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-workload-socket\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.902473 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.902378    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-workload-socket\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.902473 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.902435    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-envoy\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.902473 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.902469    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-podinfo\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.903082 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.902512    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-data\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.903082 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.902612    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-credential-socket\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.903082 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.902654    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-workload-certs\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.903082 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.902682    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3b144d5-b456-418d-948d-1b67a9cbd68e-istiod-ca-cert\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.903082 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.902744    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-token\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.903082 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.902775    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktfw5\" (UniqueName: \"kubernetes.io/projected/d3b144d5-b456-418d-948d-1b67a9cbd68e-kube-api-access-ktfw5\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.903082 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.902844    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-workload-socket\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.903478 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.903161    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-workload-certs\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.903478 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.903337    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-credential-socket\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.903478 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.903461    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-data\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.903772 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.903751    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3b144d5-b456-418d-948d-1b67a9cbd68e-istiod-ca-cert\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.905068 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.905045    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-envoy\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.905626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.905603    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-podinfo\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.910930 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.910908    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktfw5\" (UniqueName: \"kubernetes.io/projected/d3b144d5-b456-418d-948d-1b67a9cbd68e-kube-api-access-ktfw5\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.911138 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.911000    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-token\") pod \"e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") " pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:33.971744 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:33.971707    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:34.253595 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:34.253562    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"]
Sep 06 05:11:34.254061 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:11:34.254037    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b144d5_b456_418d_948d_1b67a9cbd68e.slice/crio-c66e6a48177307fa9826564fbbe2d44869e2838835c98cfad02c770bacea3807 WatchSource:0}: Error finding container c66e6a48177307fa9826564fbbe2d44869e2838835c98cfad02c770bacea3807: Status 404 returned error can't find the container with id c66e6a48177307fa9826564fbbe2d44869e2838835c98cfad02c770bacea3807
Sep 06 05:11:34.256426 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:34.256391    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:34.256522 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:34.256455    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:34.256522 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:34.256486    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:34.645671 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:34.645636    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2" event={"ID":"d3b144d5-b456-418d-948d-1b67a9cbd68e","Type":"ContainerStarted","Data":"06f7ecc9065e40d475ed4a2c94eab160daeab3e5c1a91715c6a09fb3d3b5025f"}
Sep 06 05:11:34.645845 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:34.645701    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2" event={"ID":"d3b144d5-b456-418d-948d-1b67a9cbd68e","Type":"ContainerStarted","Data":"c66e6a48177307fa9826564fbbe2d44869e2838835c98cfad02c770bacea3807"}
Sep 06 05:11:34.687658 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:34.687595    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2" podStartSLOduration=1.687573842 podStartE2EDuration="1.687573842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:11:34.685940887 +0000 UTC m=+962.055396128" watchObservedRunningTime="2026-09-06 05:11:34.687573842 +0000 UTC m=+962.057029092"
Sep 06 05:11:34.972337 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:34.972252    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:34.977981 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:34.977945    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:35.154882 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:35.154842    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24b12420-bd61-4b5d-ae49-862aae2470d9" path="/var/lib/kubelet/pods/24b12420-bd61-4b5d-ae49-862aae2470d9/volumes"
Sep 06 05:11:35.155344 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:35.155325    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9259b7e0-fe7b-462e-bc7b-f651b027fc44" path="/var/lib/kubelet/pods/9259b7e0-fe7b-462e-bc7b-f651b027fc44/volumes"
Sep 06 05:11:35.653054 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:35.653001    2578 generic.go:356] "Generic (PLEG): container finished" podID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" containerID="ab710e099ae8b8a7a8bfd39304f475903b9879839bf0846073d92e6f048e6727" exitCode=0
Sep 06 05:11:35.653250 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:35.653056    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm" event={"ID":"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312","Type":"ContainerDied","Data":"ab710e099ae8b8a7a8bfd39304f475903b9879839bf0846073d92e6f048e6727"}
Sep 06 05:11:35.653569 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:35.653551    2578 scope.go:117] "RemoveContainer" containerID="ab710e099ae8b8a7a8bfd39304f475903b9879839bf0846073d92e6f048e6727"
Sep 06 05:11:35.654624 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:35.654599    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:35.654687 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:35.654673    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:35.654724 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:35.654708    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:35.978642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:35.978605    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:36.141044 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:36.140985    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:36.241449 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:36.241369    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:36.596005 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:36.595907    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:36.660610 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:36.660574    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm" event={"ID":"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312","Type":"ContainerStarted","Data":"f58c3b85566824b52ed68806973f645798c7319a343378559e8e6c886f796068"}
Sep 06 05:11:36.660832 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:36.660815    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:36.661737 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:36.661711    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:11:36.667755 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:36.667339    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:36.815672 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:36.815627    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:36.995477 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:36.995437    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:36.995971 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:36.995560    2578 patch_prober.go:28] interesting pod/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 06 05:11:36.995971 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:36.995612    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm" podUID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 06 05:11:37.165081 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.165050    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:37.261436 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.261356    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:37.268598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.268571    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"]
Sep 06 05:11:37.271233 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.271213    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"
Sep 06 05:11:37.273525 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.273500    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-85214949-dockercfg-fhvtt\""
Sep 06 05:11:37.273625 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.273561    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-85214949\""
Sep 06 05:11:37.285066 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.285045    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"]
Sep 06 05:11:37.287501 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.287481    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"]
Sep 06 05:11:37.315041 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.311038    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"]
Sep 06 05:11:37.315041 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.311162    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"
Sep 06 05:11:37.351785 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.351754    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:37.440405 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.440370    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae0c19b2-340c-4183-b687-de9a508e8659-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj\" (UID: \"ae0c19b2-340c-4183-b687-de9a508e8659\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"
Sep 06 05:11:37.440581 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.440422    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdsrc\" (UniqueName: \"kubernetes.io/projected/ed5c83c9-3863-49ff-ac01-ba3445dec911-kube-api-access-jdsrc\") pod \"payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm\" (UID: \"ed5c83c9-3863-49ff-ac01-ba3445dec911\") " pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"
Sep 06 05:11:37.440581 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.440468    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsl79\" (UniqueName: \"kubernetes.io/projected/ae0c19b2-340c-4183-b687-de9a508e8659-kube-api-access-xsl79\") pod \"payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj\" (UID: \"ae0c19b2-340c-4183-b687-de9a508e8659\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"
Sep 06 05:11:37.440581 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.440491    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed5c83c9-3863-49ff-ac01-ba3445dec911-plugins-config-volume\") pod \"payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm\" (UID: \"ed5c83c9-3863-49ff-ac01-ba3445dec911\") " pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"
Sep 06 05:11:37.542188 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.542097    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae0c19b2-340c-4183-b687-de9a508e8659-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj\" (UID: \"ae0c19b2-340c-4183-b687-de9a508e8659\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"
Sep 06 05:11:37.542188 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.542170    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdsrc\" (UniqueName: \"kubernetes.io/projected/ed5c83c9-3863-49ff-ac01-ba3445dec911-kube-api-access-jdsrc\") pod \"payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm\" (UID: \"ed5c83c9-3863-49ff-ac01-ba3445dec911\") " pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"
Sep 06 05:11:37.542421 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.542241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsl79\" (UniqueName: \"kubernetes.io/projected/ae0c19b2-340c-4183-b687-de9a508e8659-kube-api-access-xsl79\") pod \"payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj\" (UID: \"ae0c19b2-340c-4183-b687-de9a508e8659\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"
Sep 06 05:11:37.542421 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.542279    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed5c83c9-3863-49ff-ac01-ba3445dec911-plugins-config-volume\") pod \"payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm\" (UID: \"ed5c83c9-3863-49ff-ac01-ba3445dec911\") " pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"
Sep 06 05:11:37.542852 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.542830    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae0c19b2-340c-4183-b687-de9a508e8659-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj\" (UID: \"ae0c19b2-340c-4183-b687-de9a508e8659\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"
Sep 06 05:11:37.543199 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.543178    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed5c83c9-3863-49ff-ac01-ba3445dec911-plugins-config-volume\") pod \"payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm\" (UID: \"ed5c83c9-3863-49ff-ac01-ba3445dec911\") " pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"
Sep 06 05:11:37.549713 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.549690    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsl79\" (UniqueName: \"kubernetes.io/projected/ae0c19b2-340c-4183-b687-de9a508e8659-kube-api-access-xsl79\") pod \"payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj\" (UID: \"ae0c19b2-340c-4183-b687-de9a508e8659\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"
Sep 06 05:11:37.549983 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.549960    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdsrc\" (UniqueName: \"kubernetes.io/projected/ed5c83c9-3863-49ff-ac01-ba3445dec911-kube-api-access-jdsrc\") pod \"payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm\" (UID: \"ed5c83c9-3863-49ff-ac01-ba3445dec911\") " pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"
Sep 06 05:11:37.581867 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.581832    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"
Sep 06 05:11:37.632658 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.632610    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"
Sep 06 05:11:37.883581 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.883551    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"]
Sep 06 05:11:37.885942 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:11:37.885909    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae0c19b2_340c_4183_b687_de9a508e8659.slice/crio-345b11d6a692ce0ae76e5ef54edbff26cb8034a5d439f7661f90f22094023098 WatchSource:0}: Error finding container 345b11d6a692ce0ae76e5ef54edbff26cb8034a5d439f7661f90f22094023098: Status 404 returned error can't find the container with id 345b11d6a692ce0ae76e5ef54edbff26cb8034a5d439f7661f90f22094023098
Sep 06 05:11:37.946384 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:37.946360    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"]
Sep 06 05:11:37.949494 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:11:37.949332    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded5c83c9_3863_49ff_ac01_ba3445dec911.slice/crio-a4ab66298f289b6ad871d60af698593c7b5bd7384253f8d357c137f4d54be405 WatchSource:0}: Error finding container a4ab66298f289b6ad871d60af698593c7b5bd7384253f8d357c137f4d54be405: Status 404 returned error can't find the container with id a4ab66298f289b6ad871d60af698593c7b5bd7384253f8d357c137f4d54be405
Sep 06 05:11:38.001801 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:38.001763    2578 patch_prober.go:28] interesting pod/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 06 05:11:38.002297 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:38.001828    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm" podUID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 06 05:11:38.685420 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:38.685383    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm" event={"ID":"ed5c83c9-3863-49ff-ac01-ba3445dec911","Type":"ContainerStarted","Data":"ec0c021914b7b85b0273be6d6e79d779f78fa67f395d55a03c7320b56e7e55b0"}
Sep 06 05:11:38.685630 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:38.685461    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm" event={"ID":"ed5c83c9-3863-49ff-ac01-ba3445dec911","Type":"ContainerStarted","Data":"a4ab66298f289b6ad871d60af698593c7b5bd7384253f8d357c137f4d54be405"}
Sep 06 05:11:38.685630 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:38.685519    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"
Sep 06 05:11:38.687216 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:38.687190    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj" event={"ID":"ae0c19b2-340c-4183-b687-de9a508e8659","Type":"ContainerStarted","Data":"602b5768b6278717936a61551d0b25e1e22929a2dc271b9f45a364b0b5edf326"}
Sep 06 05:11:38.687335 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:38.687231    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj" event={"ID":"ae0c19b2-340c-4183-b687-de9a508e8659","Type":"ContainerStarted","Data":"345b11d6a692ce0ae76e5ef54edbff26cb8034a5d439f7661f90f22094023098"}
Sep 06 05:11:38.687335 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:38.687291    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"
Sep 06 05:11:38.713924 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:38.713860    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm" podStartSLOduration=1.7138380039999999 podStartE2EDuration="1.713838004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:11:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:11:38.712772297 +0000 UTC m=+966.082227536" watchObservedRunningTime="2026-09-06 05:11:38.713838004 +0000 UTC m=+966.083293251"
Sep 06 05:11:38.728598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:38.728547    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj" podStartSLOduration=1.728529828 podStartE2EDuration="1.728529828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:11:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:11:38.726933859 +0000 UTC m=+966.096389106" watchObservedRunningTime="2026-09-06 05:11:38.728529828 +0000 UTC m=+966.097985074"
Sep 06 05:11:38.995812 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:38.995706    2578 patch_prober.go:28] interesting pod/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 06 05:11:38.995812 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:38.995778    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm" podUID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 06 05:11:39.995735 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:39.995695    2578 patch_prober.go:28] interesting pod/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 06 05:11:39.996363 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:39.996320    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm" podUID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 06 05:11:40.996225 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:40.996182    2578 patch_prober.go:28] interesting pod/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 06 05:11:40.996701 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:40.996248    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm" podUID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 06 05:11:41.999910 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:41.999876    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:42.000414 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:42.000393    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:42.001195 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:42.001176    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:11:48.893263 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:48.893228    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:49.698977 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:49.698941    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"
Sep 06 05:11:49.699412 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:49.699386    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"
Sep 06 05:11:54.826250 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:54.826216    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:54.925899 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:54.925865    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:11:59.815349 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:59.815317    2578 generic.go:356] "Generic (PLEG): container finished" podID="d3b144d5-b456-418d-948d-1b67a9cbd68e" containerID="06f7ecc9065e40d475ed4a2c94eab160daeab3e5c1a91715c6a09fb3d3b5025f" exitCode=0
Sep 06 05:11:59.815752 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:59.815389    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2" event={"ID":"d3b144d5-b456-418d-948d-1b67a9cbd68e","Type":"ContainerDied","Data":"06f7ecc9065e40d475ed4a2c94eab160daeab3e5c1a91715c6a09fb3d3b5025f"}
Sep 06 05:11:59.815817 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:59.815803    2578 scope.go:117] "RemoveContainer" containerID="06f7ecc9065e40d475ed4a2c94eab160daeab3e5c1a91715c6a09fb3d3b5025f"
Sep 06 05:11:59.816964 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:59.816937    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:59.817056 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:59.817004    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:11:59.817056 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:11:59.817051    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:12:00.827964 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:00.827920    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2" event={"ID":"d3b144d5-b456-418d-948d-1b67a9cbd68e","Type":"ContainerStarted","Data":"af332f43a69bfecb281ce7b0d57d02be0b8c13cd02314451c42558f6702d222b"}
Sep 06 05:12:00.972573 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:00.972542    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:12:00.972762 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:00.972741    2578 patch_prober.go:28] interesting pod/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 06 05:12:00.972821 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:00.972786    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2" podUID="d3b144d5-b456-418d-948d-1b67a9cbd68e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 06 05:12:01.972362 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:01.972317    2578 patch_prober.go:28] interesting pod/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 06 05:12:01.972743 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:01.972388    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2" podUID="d3b144d5-b456-418d-948d-1b67a9cbd68e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 06 05:12:02.973104 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:02.973069    2578 patch_prober.go:28] interesting pod/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 06 05:12:02.973556 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:02.973131    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2" podUID="d3b144d5-b456-418d-948d-1b67a9cbd68e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 06 05:12:03.972244 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:03.972202    2578 patch_prober.go:28] interesting pod/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 06 05:12:03.972415 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:03.972258    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2" podUID="d3b144d5-b456-418d-948d-1b67a9cbd68e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 06 05:12:04.972259 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:04.972220    2578 patch_prober.go:28] interesting pod/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 06 05:12:04.972646 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:04.972279    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2" podUID="d3b144d5-b456-418d-948d-1b67a9cbd68e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 06 05:12:05.976205 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:05.976174    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:12:05.976734 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:05.976442    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:12:05.977066 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:05.977048    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:12:07.676264 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:07.676227    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:07.700010 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:07.699974    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"]
Sep 06 05:12:07.700290 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:07.700249    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm" podUID="ed5c83c9-3863-49ff-ac01-ba3445dec911" containerName="payload-processing" containerID="cri-o://ec0c021914b7b85b0273be6d6e79d779f78fa67f395d55a03c7320b56e7e55b0" gracePeriod=30
Sep 06 05:12:07.705805 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:07.705777    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"]
Sep 06 05:12:07.706055 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:07.706029    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj" podUID="ae0c19b2-340c-4183-b687-de9a508e8659" containerName="payload-pre-processing" containerID="cri-o://602b5768b6278717936a61551d0b25e1e22929a2dc271b9f45a364b0b5edf326" gracePeriod=30
Sep 06 05:12:07.736902 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:07.736867    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:07.799648 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:07.799585    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:07.859340 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:07.859169    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:07.876649 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:07.876489    2578 generic.go:356] "Generic (PLEG): container finished" podID="ed5c83c9-3863-49ff-ac01-ba3445dec911" containerID="ec0c021914b7b85b0273be6d6e79d779f78fa67f395d55a03c7320b56e7e55b0" exitCode=0
Sep 06 05:12:07.876649 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:07.876565    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm" event={"ID":"ed5c83c9-3863-49ff-ac01-ba3445dec911","Type":"ContainerDied","Data":"ec0c021914b7b85b0273be6d6e79d779f78fa67f395d55a03c7320b56e7e55b0"}
Sep 06 05:12:07.878604 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:07.878297    2578 generic.go:356] "Generic (PLEG): container finished" podID="ae0c19b2-340c-4183-b687-de9a508e8659" containerID="602b5768b6278717936a61551d0b25e1e22929a2dc271b9f45a364b0b5edf326" exitCode=0
Sep 06 05:12:07.878604 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:07.878382    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj" event={"ID":"ae0c19b2-340c-4183-b687-de9a508e8659","Type":"ContainerDied","Data":"602b5768b6278717936a61551d0b25e1e22929a2dc271b9f45a364b0b5edf326"}
Sep 06 05:12:07.971247 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:07.971142    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"]
Sep 06 05:12:07.990489 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:07.990386    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:08.014786 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.014762    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"
Sep 06 05:12:08.031683 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.031650    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"
Sep 06 05:12:08.162175 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.162138    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsl79\" (UniqueName: \"kubernetes.io/projected/ae0c19b2-340c-4183-b687-de9a508e8659-kube-api-access-xsl79\") pod \"ae0c19b2-340c-4183-b687-de9a508e8659\" (UID: \"ae0c19b2-340c-4183-b687-de9a508e8659\") "
Sep 06 05:12:08.162360 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.162182    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae0c19b2-340c-4183-b687-de9a508e8659-plugins-config-volume\") pod \"ae0c19b2-340c-4183-b687-de9a508e8659\" (UID: \"ae0c19b2-340c-4183-b687-de9a508e8659\") "
Sep 06 05:12:08.162360 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.162208    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdsrc\" (UniqueName: \"kubernetes.io/projected/ed5c83c9-3863-49ff-ac01-ba3445dec911-kube-api-access-jdsrc\") pod \"ed5c83c9-3863-49ff-ac01-ba3445dec911\" (UID: \"ed5c83c9-3863-49ff-ac01-ba3445dec911\") "
Sep 06 05:12:08.162360 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.162254    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed5c83c9-3863-49ff-ac01-ba3445dec911-plugins-config-volume\") pod \"ed5c83c9-3863-49ff-ac01-ba3445dec911\" (UID: \"ed5c83c9-3863-49ff-ac01-ba3445dec911\") "
Sep 06 05:12:08.162602 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.162569    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae0c19b2-340c-4183-b687-de9a508e8659-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ae0c19b2-340c-4183-b687-de9a508e8659" (UID: "ae0c19b2-340c-4183-b687-de9a508e8659"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:12:08.162745 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.162617    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed5c83c9-3863-49ff-ac01-ba3445dec911-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed5c83c9-3863-49ff-ac01-ba3445dec911" (UID: "ed5c83c9-3863-49ff-ac01-ba3445dec911"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:12:08.162745 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.162718    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae0c19b2-340c-4183-b687-de9a508e8659-plugins-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:08.162745 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.162737    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed5c83c9-3863-49ff-ac01-ba3445dec911-plugins-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:08.164322 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.164302    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae0c19b2-340c-4183-b687-de9a508e8659-kube-api-access-xsl79" (OuterVolumeSpecName: "kube-api-access-xsl79") pod "ae0c19b2-340c-4183-b687-de9a508e8659" (UID: "ae0c19b2-340c-4183-b687-de9a508e8659"). InnerVolumeSpecName "kube-api-access-xsl79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:12:08.164456 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.164437    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed5c83c9-3863-49ff-ac01-ba3445dec911-kube-api-access-jdsrc" (OuterVolumeSpecName: "kube-api-access-jdsrc") pod "ed5c83c9-3863-49ff-ac01-ba3445dec911" (UID: "ed5c83c9-3863-49ff-ac01-ba3445dec911"). InnerVolumeSpecName "kube-api-access-jdsrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:12:08.264288 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.264206    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsl79\" (UniqueName: \"kubernetes.io/projected/ae0c19b2-340c-4183-b687-de9a508e8659-kube-api-access-xsl79\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:08.264288 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.264236    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdsrc\" (UniqueName: \"kubernetes.io/projected/ed5c83c9-3863-49ff-ac01-ba3445dec911-kube-api-access-jdsrc\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:08.422138 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.422097    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:08.682069 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.682039    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"]
Sep 06 05:12:08.682460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.682446    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed5c83c9-3863-49ff-ac01-ba3445dec911" containerName="payload-processing"
Sep 06 05:12:08.682460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.682460    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed5c83c9-3863-49ff-ac01-ba3445dec911" containerName="payload-processing"
Sep 06 05:12:08.682548 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.682470    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae0c19b2-340c-4183-b687-de9a508e8659" containerName="payload-pre-processing"
Sep 06 05:12:08.682548 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.682477    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae0c19b2-340c-4183-b687-de9a508e8659" containerName="payload-pre-processing"
Sep 06 05:12:08.682611 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.682557    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae0c19b2-340c-4183-b687-de9a508e8659" containerName="payload-pre-processing"
Sep 06 05:12:08.682611 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.682568    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed5c83c9-3863-49ff-ac01-ba3445dec911" containerName="payload-processing"
Sep 06 05:12:08.686761 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.686743    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.688879 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.688859    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a4f2603a-openshift-default-dockercfg-7hjg2\""
Sep 06 05:12:08.696843 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.696327    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:08.698235 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.698211    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"]
Sep 06 05:12:08.768771 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.768731    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-podinfo\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.768771 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.768772    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-envoy\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.768975 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.768866    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-credential-socket\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.768975 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.768905    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-data\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.768975 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.768932    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2694e5e-2402-4128-8d61-bf9094ab04df-istiod-ca-cert\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.768975 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.768957    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvqlk\" (UniqueName: \"kubernetes.io/projected/e2694e5e-2402-4128-8d61-bf9094ab04df-kube-api-access-rvqlk\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.769149 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.768995    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-workload-socket\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.769149 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.769049    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-token\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.769149 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.769115    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-workload-certs\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.870658 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.870622    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-workload-certs\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.870864 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.870719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-podinfo\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.870864 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.870741    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-envoy\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.870864 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.870767    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-credential-socket\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.870864 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.870802    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-data\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.870864 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.870834    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2694e5e-2402-4128-8d61-bf9094ab04df-istiod-ca-cert\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.870864 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.870851    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvqlk\" (UniqueName: \"kubernetes.io/projected/e2694e5e-2402-4128-8d61-bf9094ab04df-kube-api-access-rvqlk\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.871242 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.870875    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-workload-socket\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.871242 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.870904    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-token\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.871242 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.871127    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-workload-certs\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.871242 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.871177    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-data\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.871459 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.871442    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-credential-socket\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.871585 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.871560    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-workload-socket\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.871713 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.871695    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2694e5e-2402-4128-8d61-bf9094ab04df-istiod-ca-cert\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.873096 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.873071    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-envoy\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.873312 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.873295    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-podinfo\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.883719 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.883699    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-token\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.883820 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.883780    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvqlk\" (UniqueName: \"kubernetes.io/projected/e2694e5e-2402-4128-8d61-bf9094ab04df-kube-api-access-rvqlk\") pod \"e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") " pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:08.885972 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.885955    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"
Sep 06 05:12:08.886087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.885980    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm" event={"ID":"ed5c83c9-3863-49ff-ac01-ba3445dec911","Type":"ContainerDied","Data":"a4ab66298f289b6ad871d60af698593c7b5bd7384253f8d357c137f4d54be405"}
Sep 06 05:12:08.886087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.886070    2578 scope.go:117] "RemoveContainer" containerID="ec0c021914b7b85b0273be6d6e79d779f78fa67f395d55a03c7320b56e7e55b0"
Sep 06 05:12:08.887514 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.887457    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"
Sep 06 05:12:08.887514 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.887482    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj" event={"ID":"ae0c19b2-340c-4183-b687-de9a508e8659","Type":"ContainerDied","Data":"345b11d6a692ce0ae76e5ef54edbff26cb8034a5d439f7661f90f22094023098"}
Sep 06 05:12:08.888107 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.887756    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2" podUID="d3b144d5-b456-418d-948d-1b67a9cbd68e" containerName="istio-proxy" containerID="cri-o://af332f43a69bfecb281ce7b0d57d02be0b8c13cd02314451c42558f6702d222b" gracePeriod=30
Sep 06 05:12:08.905545 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.905523    2578 scope.go:117] "RemoveContainer" containerID="602b5768b6278717936a61551d0b25e1e22929a2dc271b9f45a364b0b5edf326"
Sep 06 05:12:08.926213 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.926188    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"]
Sep 06 05:12:08.929133 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.929108    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-85214949-77846f5b94-77qmj"]
Sep 06 05:12:08.938323 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.938272    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"]
Sep 06 05:12:08.941908 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.941888    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-85214949-89c54bf66-xt7bm"]
Sep 06 05:12:08.998549 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:08.998517    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:09.147259 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:09.147225    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:09.153561 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:09.153537    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae0c19b2-340c-4183-b687-de9a508e8659" path="/var/lib/kubelet/pods/ae0c19b2-340c-4183-b687-de9a508e8659/volumes"
Sep 06 05:12:09.153923 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:09.153910    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed5c83c9-3863-49ff-ac01-ba3445dec911" path="/var/lib/kubelet/pods/ed5c83c9-3863-49ff-ac01-ba3445dec911/volumes"
Sep 06 05:12:09.275467 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:09.275436    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"]
Sep 06 05:12:09.277548 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:12:09.277521    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2694e5e_2402_4128_8d61_bf9094ab04df.slice/crio-df6870d3a3e9ccd4bd5af7ccae8a809cc3f8c01426438c40314ad350fbfca50c WatchSource:0}: Error finding container df6870d3a3e9ccd4bd5af7ccae8a809cc3f8c01426438c40314ad350fbfca50c: Status 404 returned error can't find the container with id df6870d3a3e9ccd4bd5af7ccae8a809cc3f8c01426438c40314ad350fbfca50c
Sep 06 05:12:09.279813 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:09.279784    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:12:09.279885 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:09.279846    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:12:09.279885 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:09.279874    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:12:09.897941 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:09.897907    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9" event={"ID":"e2694e5e-2402-4128-8d61-bf9094ab04df","Type":"ContainerStarted","Data":"8f71d1065b23e5cfc35204a1ffdc99892bb0eb81834abd513cd807a1501ba0a2"}
Sep 06 05:12:09.898336 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:09.897955    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9" event={"ID":"e2694e5e-2402-4128-8d61-bf9094ab04df","Type":"ContainerStarted","Data":"df6870d3a3e9ccd4bd5af7ccae8a809cc3f8c01426438c40314ad350fbfca50c"}
Sep 06 05:12:09.917306 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:09.917242    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9" podStartSLOduration=1.917223168 podStartE2EDuration="1.917223168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:12:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:12:09.913530991 +0000 UTC m=+997.282986249" watchObservedRunningTime="2026-09-06 05:12:09.917223168 +0000 UTC m=+997.286678417"
Sep 06 05:12:09.998985 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:09.998950    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:11.004377 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:11.004348    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:11.321151 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:11.321065    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:11.414798 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:11.414770    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:11.908350 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:11.908313    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:11.909386 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:11.909365    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:12.543878 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:12.543840    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:12.641562 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:12.641519    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:12.967644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:12.967615    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:13.066425 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:13.066382    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:13.144552 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:13.144517    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:14.137869 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.137841    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:12:14.226383 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.226294    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3b144d5-b456-418d-948d-1b67a9cbd68e-istiod-ca-cert\") pod \"d3b144d5-b456-418d-948d-1b67a9cbd68e\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") "
Sep 06 05:12:14.226383 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.226342    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-data\") pod \"d3b144d5-b456-418d-948d-1b67a9cbd68e\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") "
Sep 06 05:12:14.226596 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.226391    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-workload-certs\") pod \"d3b144d5-b456-418d-948d-1b67a9cbd68e\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") "
Sep 06 05:12:14.226596 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.226418    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-credential-socket\") pod \"d3b144d5-b456-418d-948d-1b67a9cbd68e\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") "
Sep 06 05:12:14.226596 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.226484    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-envoy\") pod \"d3b144d5-b456-418d-948d-1b67a9cbd68e\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") "
Sep 06 05:12:14.226596 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.226509    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktfw5\" (UniqueName: \"kubernetes.io/projected/d3b144d5-b456-418d-948d-1b67a9cbd68e-kube-api-access-ktfw5\") pod \"d3b144d5-b456-418d-948d-1b67a9cbd68e\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") "
Sep 06 05:12:14.226596 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.226527    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-podinfo\") pod \"d3b144d5-b456-418d-948d-1b67a9cbd68e\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") "
Sep 06 05:12:14.226596 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.226555    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-token\") pod \"d3b144d5-b456-418d-948d-1b67a9cbd68e\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") "
Sep 06 05:12:14.226596 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.226571    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-workload-socket\") pod \"d3b144d5-b456-418d-948d-1b67a9cbd68e\" (UID: \"d3b144d5-b456-418d-948d-1b67a9cbd68e\") "
Sep 06 05:12:14.226963 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.226686    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d3b144d5-b456-418d-948d-1b67a9cbd68e" (UID: "d3b144d5-b456-418d-948d-1b67a9cbd68e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:12:14.226963 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.226704    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-data" (OuterVolumeSpecName: "istio-data") pod "d3b144d5-b456-418d-948d-1b67a9cbd68e" (UID: "d3b144d5-b456-418d-948d-1b67a9cbd68e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:12:14.226963 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.226747    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3b144d5-b456-418d-948d-1b67a9cbd68e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d3b144d5-b456-418d-948d-1b67a9cbd68e" (UID: "d3b144d5-b456-418d-948d-1b67a9cbd68e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:12:14.226963 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.226900    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d3b144d5-b456-418d-948d-1b67a9cbd68e" (UID: "d3b144d5-b456-418d-948d-1b67a9cbd68e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:12:14.226963 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.226916    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d3b144d5-b456-418d-948d-1b67a9cbd68e" (UID: "d3b144d5-b456-418d-948d-1b67a9cbd68e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:12:14.227283 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.227047    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-workload-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:14.227283 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.227066    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-credential-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:14.227283 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.227079    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-workload-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:14.227283 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.227093    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3b144d5-b456-418d-948d-1b67a9cbd68e-istiod-ca-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:14.227283 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.227105    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-data\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:14.228909 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.228877    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-token" (OuterVolumeSpecName: "istio-token") pod "d3b144d5-b456-418d-948d-1b67a9cbd68e" (UID: "d3b144d5-b456-418d-948d-1b67a9cbd68e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:12:14.229041 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.228959    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3b144d5-b456-418d-948d-1b67a9cbd68e-kube-api-access-ktfw5" (OuterVolumeSpecName: "kube-api-access-ktfw5") pod "d3b144d5-b456-418d-948d-1b67a9cbd68e" (UID: "d3b144d5-b456-418d-948d-1b67a9cbd68e"). InnerVolumeSpecName "kube-api-access-ktfw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:12:14.229239 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.229220    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d3b144d5-b456-418d-948d-1b67a9cbd68e" (UID: "d3b144d5-b456-418d-948d-1b67a9cbd68e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:12:14.229320 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.229235    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d3b144d5-b456-418d-948d-1b67a9cbd68e" (UID: "d3b144d5-b456-418d-948d-1b67a9cbd68e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:12:14.328077 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.328046    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-envoy\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:14.328077 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.328071    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktfw5\" (UniqueName: \"kubernetes.io/projected/d3b144d5-b456-418d-948d-1b67a9cbd68e-kube-api-access-ktfw5\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:14.328077 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.328080    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-podinfo\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:14.328077 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.328088    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3b144d5-b456-418d-948d-1b67a9cbd68e-istio-token\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:14.925554 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.925515    2578 generic.go:356] "Generic (PLEG): container finished" podID="d3b144d5-b456-418d-948d-1b67a9cbd68e" containerID="af332f43a69bfecb281ce7b0d57d02be0b8c13cd02314451c42558f6702d222b" exitCode=0
Sep 06 05:12:14.925701 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.925566    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"
Sep 06 05:12:14.925701 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.925573    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2" event={"ID":"d3b144d5-b456-418d-948d-1b67a9cbd68e","Type":"ContainerDied","Data":"af332f43a69bfecb281ce7b0d57d02be0b8c13cd02314451c42558f6702d222b"}
Sep 06 05:12:14.925701 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.925623    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2" event={"ID":"d3b144d5-b456-418d-948d-1b67a9cbd68e","Type":"ContainerDied","Data":"c66e6a48177307fa9826564fbbe2d44869e2838835c98cfad02c770bacea3807"}
Sep 06 05:12:14.925701 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.925647    2578 scope.go:117] "RemoveContainer" containerID="af332f43a69bfecb281ce7b0d57d02be0b8c13cd02314451c42558f6702d222b"
Sep 06 05:12:14.935351 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.935334    2578 scope.go:117] "RemoveContainer" containerID="06f7ecc9065e40d475ed4a2c94eab160daeab3e5c1a91715c6a09fb3d3b5025f"
Sep 06 05:12:14.943656 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.943629    2578 scope.go:117] "RemoveContainer" containerID="af332f43a69bfecb281ce7b0d57d02be0b8c13cd02314451c42558f6702d222b"
Sep 06 05:12:14.944065 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:12:14.944037    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af332f43a69bfecb281ce7b0d57d02be0b8c13cd02314451c42558f6702d222b\": container with ID starting with af332f43a69bfecb281ce7b0d57d02be0b8c13cd02314451c42558f6702d222b not found: ID does not exist" containerID="af332f43a69bfecb281ce7b0d57d02be0b8c13cd02314451c42558f6702d222b"
Sep 06 05:12:14.944154 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.944075    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af332f43a69bfecb281ce7b0d57d02be0b8c13cd02314451c42558f6702d222b"} err="failed to get container status \"af332f43a69bfecb281ce7b0d57d02be0b8c13cd02314451c42558f6702d222b\": rpc error: code = NotFound desc = could not find container \"af332f43a69bfecb281ce7b0d57d02be0b8c13cd02314451c42558f6702d222b\": container with ID starting with af332f43a69bfecb281ce7b0d57d02be0b8c13cd02314451c42558f6702d222b not found: ID does not exist"
Sep 06 05:12:14.944154 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.944094    2578 scope.go:117] "RemoveContainer" containerID="06f7ecc9065e40d475ed4a2c94eab160daeab3e5c1a91715c6a09fb3d3b5025f"
Sep 06 05:12:14.944380 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:12:14.944356    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06f7ecc9065e40d475ed4a2c94eab160daeab3e5c1a91715c6a09fb3d3b5025f\": container with ID starting with 06f7ecc9065e40d475ed4a2c94eab160daeab3e5c1a91715c6a09fb3d3b5025f not found: ID does not exist" containerID="06f7ecc9065e40d475ed4a2c94eab160daeab3e5c1a91715c6a09fb3d3b5025f"
Sep 06 05:12:14.944438 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.944390    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06f7ecc9065e40d475ed4a2c94eab160daeab3e5c1a91715c6a09fb3d3b5025f"} err="failed to get container status \"06f7ecc9065e40d475ed4a2c94eab160daeab3e5c1a91715c6a09fb3d3b5025f\": rpc error: code = NotFound desc = could not find container \"06f7ecc9065e40d475ed4a2c94eab160daeab3e5c1a91715c6a09fb3d3b5025f\": container with ID starting with 06f7ecc9065e40d475ed4a2c94eab160daeab3e5c1a91715c6a09fb3d3b5025f not found: ID does not exist"
Sep 06 05:12:14.946284 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.946266    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"]
Sep 06 05:12:14.949439 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:14.949417    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-85214949-gw-openshift-default-694487b56d-ccxz2"]
Sep 06 05:12:15.158642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:15.158605    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3b144d5-b456-418d-948d-1b67a9cbd68e" path="/var/lib/kubelet/pods/d3b144d5-b456-418d-948d-1b67a9cbd68e/volumes"
Sep 06 05:12:16.833474 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:16.833440    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"]
Sep 06 05:12:16.833862 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:16.833849    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3b144d5-b456-418d-948d-1b67a9cbd68e" containerName="istio-proxy"
Sep 06 05:12:16.833901 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:16.833864    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3b144d5-b456-418d-948d-1b67a9cbd68e" containerName="istio-proxy"
Sep 06 05:12:16.833951 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:16.833942    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3b144d5-b456-418d-948d-1b67a9cbd68e" containerName="istio-proxy"
Sep 06 05:12:16.833986 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:16.833951    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3b144d5-b456-418d-948d-1b67a9cbd68e" containerName="istio-proxy"
Sep 06 05:12:16.838324 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:16.838307    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:16.840500 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:16.840481    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-0cd5b238-kserve-self-signed-certs\""
Sep 06 05:12:16.845995 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:16.845973    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"]
Sep 06 05:12:16.954499 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:16.954464    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-model-cache\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:16.954668 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:16.954506    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-home\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:16.954668 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:16.954545    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-dshm\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:16.954668 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:16.954573    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p58nb\" (UniqueName: \"kubernetes.io/projected/6b4d83b3-b96d-418a-9ca5-422834a553d6-kube-api-access-p58nb\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:16.954668 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:16.954595    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-tmp-dir\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:16.954809 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:16.954720    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b4d83b3-b96d-418a-9ca5-422834a553d6-tls-certs\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:17.055959 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.055914    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b4d83b3-b96d-418a-9ca5-422834a553d6-tls-certs\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:17.056182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.055995    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-model-cache\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:17.056182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.056061    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-home\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:17.056182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.056111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-dshm\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:17.056182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.056134    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p58nb\" (UniqueName: \"kubernetes.io/projected/6b4d83b3-b96d-418a-9ca5-422834a553d6-kube-api-access-p58nb\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:17.056182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.056163    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-tmp-dir\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:17.056524 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.056502    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-model-cache\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:17.056588 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.056565    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-tmp-dir\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:17.056624 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.056586    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-home\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:17.058353 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.058333    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-dshm\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:17.058535 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.058517    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b4d83b3-b96d-418a-9ca5-422834a553d6-tls-certs\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:17.063830 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.063806    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p58nb\" (UniqueName: \"kubernetes.io/projected/6b4d83b3-b96d-418a-9ca5-422834a553d6-kube-api-access-p58nb\") pod \"e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") " pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:17.147964 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.147933    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:17.431479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.431451    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"]
Sep 06 05:12:17.433648 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:12:17.433613    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b4d83b3_b96d_418a_9ca5_422834a553d6.slice/crio-7d3b1312f526f76268bbb4e6392ad132709853ba1c4ed7254864ce91024f429b WatchSource:0}: Error finding container 7d3b1312f526f76268bbb4e6392ad132709853ba1c4ed7254864ce91024f429b: Status 404 returned error can't find the container with id 7d3b1312f526f76268bbb4e6392ad132709853ba1c4ed7254864ce91024f429b
Sep 06 05:12:17.435902 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.435879    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:17.944358 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.944318    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd" event={"ID":"6b4d83b3-b96d-418a-9ca5-422834a553d6","Type":"ContainerStarted","Data":"a801059a4541b988c03e68b91142827f6269d4e379a8eb3c7aa45a0402ca6f4f"}
Sep 06 05:12:17.944358 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.944360    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd" event={"ID":"6b4d83b3-b96d-418a-9ca5-422834a553d6","Type":"ContainerStarted","Data":"7d3b1312f526f76268bbb4e6392ad132709853ba1c4ed7254864ce91024f429b"}
Sep 06 05:12:17.959143 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:17.959098    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd" podStartSLOduration=1.959082307 podStartE2EDuration="1.959082307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:12:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:12:17.958701825 +0000 UTC m=+1005.328157071" watchObservedRunningTime="2026-09-06 05:12:17.959082307 +0000 UTC m=+1005.328537553"
Sep 06 05:12:26.980442 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:26.980232    2578 patch_prober.go:28] interesting pod/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": read tcp 10.133.0.2:50620->10.133.0.55:15021: read: connection reset by peer" start-of-body=
Sep 06 05:12:26.980442 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:26.980319    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9" podUID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": read tcp 10.133.0.2:50620->10.133.0.55:15021: read: connection reset by peer"
Sep 06 05:12:27.153712 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:27.153644    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:27.153712 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:27.153684    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:27.160992 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:27.160971    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:28.005489 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:28.005454    2578 generic.go:356] "Generic (PLEG): container finished" podID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerID="8f71d1065b23e5cfc35204a1ffdc99892bb0eb81834abd513cd807a1501ba0a2" exitCode=0
Sep 06 05:12:28.005964 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:28.005528    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9" event={"ID":"e2694e5e-2402-4128-8d61-bf9094ab04df","Type":"ContainerDied","Data":"8f71d1065b23e5cfc35204a1ffdc99892bb0eb81834abd513cd807a1501ba0a2"}
Sep 06 05:12:28.006105 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:28.006086    2578 scope.go:117] "RemoveContainer" containerID="8f71d1065b23e5cfc35204a1ffdc99892bb0eb81834abd513cd807a1501ba0a2"
Sep 06 05:12:28.007243 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:28.007210    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:12:28.007352 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:28.007292    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:12:28.007352 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:28.007339    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:12:28.017637 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:28.017618    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:12:29.018409 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:29.018366    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9" event={"ID":"e2694e5e-2402-4128-8d61-bf9094ab04df","Type":"ContainerStarted","Data":"d0cea877b759e73e42daa49bc967d8ef4b99757a694fbe4d3837b0ef316c8eda"}
Sep 06 05:12:29.999104 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:29.999065    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:29.999306 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:29.999186    2578 patch_prober.go:28] interesting pod/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 06 05:12:29.999306 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:29.999227    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9" podUID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 06 05:12:30.999740 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:30.999305    2578 patch_prober.go:28] interesting pod/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 06 05:12:30.999740 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:30.999370    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9" podUID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 06 05:12:31.999173 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:31.999132    2578 patch_prober.go:28] interesting pod/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 06 05:12:31.999490 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:31.999448    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9" podUID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 06 05:12:32.999611 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:32.999569    2578 patch_prober.go:28] interesting pod/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 06 05:12:33.000200 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:33.000168    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9" podUID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 06 05:12:33.998963 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:33.998922    2578 patch_prober.go:28] interesting pod/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 06 05:12:33.999183 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:33.998994    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9" podUID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 06 05:12:35.003122 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:35.003089    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:35.003581 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:35.003386    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:35.004236 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:35.004214    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:43.004520 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:43.004466    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:43.067011 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:43.066968    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:43.131988 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:43.131950    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:43.193310 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:43.193275    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:44.233306 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:44.233270    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"]
Sep 06 05:12:44.233742 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:44.233501    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9" podUID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerName="istio-proxy" containerID="cri-o://d0cea877b759e73e42daa49bc967d8ef4b99757a694fbe4d3837b0ef316c8eda" gracePeriod=30
Sep 06 05:12:44.256894 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:44.256867    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:44.685880 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:44.685852    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:44.964319 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:44.964246    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:45.315165 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:45.315079    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:47.020349 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:47.020304    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:47.116228 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:47.116193    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:48.197545 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:48.197494    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:48.299391 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:48.299353    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:48.646744 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:48.646709    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:48.747318 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:48.747280    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:12:48.761182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:48.761138    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"]
Sep 06 05:12:48.761837 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:48.761815    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3b144d5-b456-418d-948d-1b67a9cbd68e" containerName="istio-proxy"
Sep 06 05:12:48.761886 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:48.761843    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3b144d5-b456-418d-948d-1b67a9cbd68e" containerName="istio-proxy"
Sep 06 05:12:48.766977 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:48.766952    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"
Sep 06 05:12:48.769654 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:48.769336    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-de5ebc7b\""
Sep 06 05:12:48.770253 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:48.770231    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-de5ebc7b-dockercfg-kwx7j\""
Sep 06 05:12:48.773421 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:48.773399    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"]
Sep 06 05:12:48.914808 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:48.914712    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4e3b11-22e3-4ea2-b0f7-578004de0116-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc\" (UID: \"7e4e3b11-22e3-4ea2-b0f7-578004de0116\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"
Sep 06 05:12:48.914808 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:48.914778    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9vlk\" (UniqueName: \"kubernetes.io/projected/7e4e3b11-22e3-4ea2-b0f7-578004de0116-kube-api-access-w9vlk\") pod \"payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc\" (UID: \"7e4e3b11-22e3-4ea2-b0f7-578004de0116\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"
Sep 06 05:12:49.016446 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.016401    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4e3b11-22e3-4ea2-b0f7-578004de0116-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc\" (UID: \"7e4e3b11-22e3-4ea2-b0f7-578004de0116\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"
Sep 06 05:12:49.016627 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.016454    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9vlk\" (UniqueName: \"kubernetes.io/projected/7e4e3b11-22e3-4ea2-b0f7-578004de0116-kube-api-access-w9vlk\") pod \"payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc\" (UID: \"7e4e3b11-22e3-4ea2-b0f7-578004de0116\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"
Sep 06 05:12:49.017466 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.017439    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4e3b11-22e3-4ea2-b0f7-578004de0116-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc\" (UID: \"7e4e3b11-22e3-4ea2-b0f7-578004de0116\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"
Sep 06 05:12:49.029786 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.029746    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9vlk\" (UniqueName: \"kubernetes.io/projected/7e4e3b11-22e3-4ea2-b0f7-578004de0116-kube-api-access-w9vlk\") pod \"payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc\" (UID: \"7e4e3b11-22e3-4ea2-b0f7-578004de0116\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"
Sep 06 05:12:49.083261 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.083226    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"
Sep 06 05:12:49.368658 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.368631    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"]
Sep 06 05:12:49.371525 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:12:49.371496    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e4e3b11_22e3_4ea2_b0f7_578004de0116.slice/crio-be3dddaf2aa4347838c7369c007bcda4647e5b180773c570a7cf656d52234198 WatchSource:0}: Error finding container be3dddaf2aa4347838c7369c007bcda4647e5b180773c570a7cf656d52234198: Status 404 returned error can't find the container with id be3dddaf2aa4347838c7369c007bcda4647e5b180773c570a7cf656d52234198
Sep 06 05:12:49.472638 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.472612    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:49.623589 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.623556    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-workload-certs\") pod \"e2694e5e-2402-4128-8d61-bf9094ab04df\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") "
Sep 06 05:12:49.623766 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.623609    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-token\") pod \"e2694e5e-2402-4128-8d61-bf9094ab04df\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") "
Sep 06 05:12:49.623766 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.623634    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-podinfo\") pod \"e2694e5e-2402-4128-8d61-bf9094ab04df\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") "
Sep 06 05:12:49.623766 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.623663    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-envoy\") pod \"e2694e5e-2402-4128-8d61-bf9094ab04df\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") "
Sep 06 05:12:49.623766 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.623709    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-credential-socket\") pod \"e2694e5e-2402-4128-8d61-bf9094ab04df\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") "
Sep 06 05:12:49.623983 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.623820    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2694e5e-2402-4128-8d61-bf9094ab04df-istiod-ca-cert\") pod \"e2694e5e-2402-4128-8d61-bf9094ab04df\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") "
Sep 06 05:12:49.623983 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.623824    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e2694e5e-2402-4128-8d61-bf9094ab04df" (UID: "e2694e5e-2402-4128-8d61-bf9094ab04df"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:12:49.623983 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.623846    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-workload-socket\") pod \"e2694e5e-2402-4128-8d61-bf9094ab04df\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") "
Sep 06 05:12:49.623983 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.623876    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-data\") pod \"e2694e5e-2402-4128-8d61-bf9094ab04df\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") "
Sep 06 05:12:49.623983 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.623903    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvqlk\" (UniqueName: \"kubernetes.io/projected/e2694e5e-2402-4128-8d61-bf9094ab04df-kube-api-access-rvqlk\") pod \"e2694e5e-2402-4128-8d61-bf9094ab04df\" (UID: \"e2694e5e-2402-4128-8d61-bf9094ab04df\") "
Sep 06 05:12:49.624289 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.624060    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e2694e5e-2402-4128-8d61-bf9094ab04df" (UID: "e2694e5e-2402-4128-8d61-bf9094ab04df"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:12:49.624352 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.624255    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e2694e5e-2402-4128-8d61-bf9094ab04df" (UID: "e2694e5e-2402-4128-8d61-bf9094ab04df"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:12:49.624690 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.624427    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-workload-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:49.624690 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.624450    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-workload-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:49.624690 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.624448    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-data" (OuterVolumeSpecName: "istio-data") pod "e2694e5e-2402-4128-8d61-bf9094ab04df" (UID: "e2694e5e-2402-4128-8d61-bf9094ab04df"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:12:49.624690 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.624463    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-credential-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:49.624690 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.624489    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2694e5e-2402-4128-8d61-bf9094ab04df-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e2694e5e-2402-4128-8d61-bf9094ab04df" (UID: "e2694e5e-2402-4128-8d61-bf9094ab04df"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:12:49.625964 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.625934    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e2694e5e-2402-4128-8d61-bf9094ab04df" (UID: "e2694e5e-2402-4128-8d61-bf9094ab04df"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:12:49.626093 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.626079    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2694e5e-2402-4128-8d61-bf9094ab04df-kube-api-access-rvqlk" (OuterVolumeSpecName: "kube-api-access-rvqlk") pod "e2694e5e-2402-4128-8d61-bf9094ab04df" (UID: "e2694e5e-2402-4128-8d61-bf9094ab04df"). InnerVolumeSpecName "kube-api-access-rvqlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:12:49.626328 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.626312    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e2694e5e-2402-4128-8d61-bf9094ab04df" (UID: "e2694e5e-2402-4128-8d61-bf9094ab04df"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:12:49.626576 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.626562    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-token" (OuterVolumeSpecName: "istio-token") pod "e2694e5e-2402-4128-8d61-bf9094ab04df" (UID: "e2694e5e-2402-4128-8d61-bf9094ab04df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:12:49.725251 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.725215    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2694e5e-2402-4128-8d61-bf9094ab04df-istiod-ca-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:49.725251 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.725247    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-data\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:49.725251 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.725256    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvqlk\" (UniqueName: \"kubernetes.io/projected/e2694e5e-2402-4128-8d61-bf9094ab04df-kube-api-access-rvqlk\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:49.725500 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.725265    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-token\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:49.725500 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.725274    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-podinfo\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:49.725500 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:49.725281    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2694e5e-2402-4128-8d61-bf9094ab04df-istio-envoy\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:12:50.136087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.136049    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc" event={"ID":"7e4e3b11-22e3-4ea2-b0f7-578004de0116","Type":"ContainerStarted","Data":"60fbe14f49cac34b6f41af768a0606de530f4ef476ca67aaae45a740cba059f9"}
Sep 06 05:12:50.136288 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.136116    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"
Sep 06 05:12:50.136288 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.136139    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc" event={"ID":"7e4e3b11-22e3-4ea2-b0f7-578004de0116","Type":"ContainerStarted","Data":"be3dddaf2aa4347838c7369c007bcda4647e5b180773c570a7cf656d52234198"}
Sep 06 05:12:50.137701 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.137678    2578 generic.go:356] "Generic (PLEG): container finished" podID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerID="d0cea877b759e73e42daa49bc967d8ef4b99757a694fbe4d3837b0ef316c8eda" exitCode=0
Sep 06 05:12:50.137824 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.137721    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"
Sep 06 05:12:50.137824 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.137728    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9" event={"ID":"e2694e5e-2402-4128-8d61-bf9094ab04df","Type":"ContainerDied","Data":"d0cea877b759e73e42daa49bc967d8ef4b99757a694fbe4d3837b0ef316c8eda"}
Sep 06 05:12:50.137824 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.137773    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9" event={"ID":"e2694e5e-2402-4128-8d61-bf9094ab04df","Type":"ContainerDied","Data":"df6870d3a3e9ccd4bd5af7ccae8a809cc3f8c01426438c40314ad350fbfca50c"}
Sep 06 05:12:50.137824 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.137803    2578 scope.go:117] "RemoveContainer" containerID="d0cea877b759e73e42daa49bc967d8ef4b99757a694fbe4d3837b0ef316c8eda"
Sep 06 05:12:50.147441 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.147424    2578 scope.go:117] "RemoveContainer" containerID="8f71d1065b23e5cfc35204a1ffdc99892bb0eb81834abd513cd807a1501ba0a2"
Sep 06 05:12:50.151652 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.151609    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc" podStartSLOduration=2.151595499 podStartE2EDuration="2.151595499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:12:50.150533786 +0000 UTC m=+1037.519989033" watchObservedRunningTime="2026-09-06 05:12:50.151595499 +0000 UTC m=+1037.521050745"
Sep 06 05:12:50.156174 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.156150    2578 scope.go:117] "RemoveContainer" containerID="d0cea877b759e73e42daa49bc967d8ef4b99757a694fbe4d3837b0ef316c8eda"
Sep 06 05:12:50.156464 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:12:50.156442    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0cea877b759e73e42daa49bc967d8ef4b99757a694fbe4d3837b0ef316c8eda\": container with ID starting with d0cea877b759e73e42daa49bc967d8ef4b99757a694fbe4d3837b0ef316c8eda not found: ID does not exist" containerID="d0cea877b759e73e42daa49bc967d8ef4b99757a694fbe4d3837b0ef316c8eda"
Sep 06 05:12:50.156508 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.156473    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0cea877b759e73e42daa49bc967d8ef4b99757a694fbe4d3837b0ef316c8eda"} err="failed to get container status \"d0cea877b759e73e42daa49bc967d8ef4b99757a694fbe4d3837b0ef316c8eda\": rpc error: code = NotFound desc = could not find container \"d0cea877b759e73e42daa49bc967d8ef4b99757a694fbe4d3837b0ef316c8eda\": container with ID starting with d0cea877b759e73e42daa49bc967d8ef4b99757a694fbe4d3837b0ef316c8eda not found: ID does not exist"
Sep 06 05:12:50.156508 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.156490    2578 scope.go:117] "RemoveContainer" containerID="8f71d1065b23e5cfc35204a1ffdc99892bb0eb81834abd513cd807a1501ba0a2"
Sep 06 05:12:50.156766 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:12:50.156747    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f71d1065b23e5cfc35204a1ffdc99892bb0eb81834abd513cd807a1501ba0a2\": container with ID starting with 8f71d1065b23e5cfc35204a1ffdc99892bb0eb81834abd513cd807a1501ba0a2 not found: ID does not exist" containerID="8f71d1065b23e5cfc35204a1ffdc99892bb0eb81834abd513cd807a1501ba0a2"
Sep 06 05:12:50.156819 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.156771    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f71d1065b23e5cfc35204a1ffdc99892bb0eb81834abd513cd807a1501ba0a2"} err="failed to get container status \"8f71d1065b23e5cfc35204a1ffdc99892bb0eb81834abd513cd807a1501ba0a2\": rpc error: code = NotFound desc = could not find container \"8f71d1065b23e5cfc35204a1ffdc99892bb0eb81834abd513cd807a1501ba0a2\": container with ID starting with 8f71d1065b23e5cfc35204a1ffdc99892bb0eb81834abd513cd807a1501ba0a2 not found: ID does not exist"
Sep 06 05:12:50.164910 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.164887    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"]
Sep 06 05:12:50.167935 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:50.167912    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a4f2603a-openshift-default-69b9d6c494-tczt9"]
Sep 06 05:12:51.153752 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:51.153724    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2694e5e-2402-4128-8d61-bf9094ab04df" path="/var/lib/kubelet/pods/e2694e5e-2402-4128-8d61-bf9094ab04df/volumes"
Sep 06 05:12:54.565783 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:12:54.565750    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:01.146746 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:01.146715    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"
Sep 06 05:13:09.166634 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:09.166595    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:12.840611 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.840573    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"]
Sep 06 05:13:12.841770 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.841738    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerName="istio-proxy"
Sep 06 05:13:12.841770 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.841770    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerName="istio-proxy"
Sep 06 05:13:12.841966 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.841784    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerName="istio-proxy"
Sep 06 05:13:12.841966 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.841793    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerName="istio-proxy"
Sep 06 05:13:12.841966 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.841932    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerName="istio-proxy"
Sep 06 05:13:12.841966 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.841946    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2694e5e-2402-4128-8d61-bf9094ab04df" containerName="istio-proxy"
Sep 06 05:13:12.845463 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.845443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:12.849578 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.848913    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-9afd7f\"/\"kube-root-ca.crt\""
Sep 06 05:13:12.849578 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.849205    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-9afd7f\"/\"autores-model-9afd7f-kserve-self-signed-certs\""
Sep 06 05:13:12.849578 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.849339    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-9afd7f\"/\"openshift-service-ca.crt\""
Sep 06 05:13:12.849578 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.849425    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-9afd7f\"/\"default-dockercfg-xltc6\""
Sep 06 05:13:12.851922 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.851897    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"]
Sep 06 05:13:12.875441 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.875399    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:12.944835 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.944797    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-tmp-dir\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:12.945058 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.944849    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6481692e-fb55-465e-8c6c-40400f7f476b-tls-certs\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:12.945058 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.944880    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s595q\" (UniqueName: \"kubernetes.io/projected/6481692e-fb55-465e-8c6c-40400f7f476b-kube-api-access-s595q\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:12.945058 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.944913    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-home\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:12.945058 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.944936    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-model-cache\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:12.945058 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:12.945000    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-dshm\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:13.045752 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:13.045707    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-tmp-dir\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:13.045752 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:13.045756    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6481692e-fb55-465e-8c6c-40400f7f476b-tls-certs\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:13.046009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:13.045884    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s595q\" (UniqueName: \"kubernetes.io/projected/6481692e-fb55-465e-8c6c-40400f7f476b-kube-api-access-s595q\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:13.046009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:13.045965    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-home\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:13.046009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:13.046028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-model-cache\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:13.046212 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:13.046084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-dshm\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:13.046212 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:13.046169    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-tmp-dir\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:13.046337 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:13.046316    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-home\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:13.046400 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:13.046385    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-model-cache\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:13.048337 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:13.048315    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-dshm\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:13.048487 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:13.048468    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6481692e-fb55-465e-8c6c-40400f7f476b-tls-certs\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:13.054086 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:13.054061    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s595q\" (UniqueName: \"kubernetes.io/projected/6481692e-fb55-465e-8c6c-40400f7f476b-kube-api-access-s595q\") pod \"autores-model-9afd7f-kserve-68894cb59f-59n46\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") " pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:13.157668 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:13.157638    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:13.431657 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:13.431615    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"]
Sep 06 05:13:13.434002 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:13:13.433974    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6481692e_fb55_465e_8c6c_40400f7f476b.slice/crio-2f6cb1b1c65aa6b4d5bea38d8bd7d0c2e8c4d1ca3eb3772a07cb9bb9987a4662 WatchSource:0}: Error finding container 2f6cb1b1c65aa6b4d5bea38d8bd7d0c2e8c4d1ca3eb3772a07cb9bb9987a4662: Status 404 returned error can't find the container with id 2f6cb1b1c65aa6b4d5bea38d8bd7d0c2e8c4d1ca3eb3772a07cb9bb9987a4662
Sep 06 05:13:14.268314 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:14.268279    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46" event={"ID":"6481692e-fb55-465e-8c6c-40400f7f476b","Type":"ContainerStarted","Data":"ee708afcaecfe6151a4c892842ee824fcae455ef1af8f134919b292eecb7dbf2"}
Sep 06 05:13:14.268698 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:14.268322    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46" event={"ID":"6481692e-fb55-465e-8c6c-40400f7f476b","Type":"ContainerStarted","Data":"2f6cb1b1c65aa6b4d5bea38d8bd7d0c2e8c4d1ca3eb3772a07cb9bb9987a4662"}
Sep 06 05:13:14.287959 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:14.287907    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46" podStartSLOduration=2.287890657 podStartE2EDuration="2.287890657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:13:14.285974868 +0000 UTC m=+1061.655430113" watchObservedRunningTime="2026-09-06 05:13:14.287890657 +0000 UTC m=+1061.657345905"
Sep 06 05:13:15.270352 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:15.270317    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:18.616893 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:18.616852    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:18.646110 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:18.646080    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"]
Sep 06 05:13:18.646344 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:18.646316    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc" podUID="7e4e3b11-22e3-4ea2-b0f7-578004de0116" containerName="payload-pre-processing" containerID="cri-o://60fbe14f49cac34b6f41af768a0606de530f4ef476ca67aaae45a740cba059f9" gracePeriod=30
Sep 06 05:13:18.717807 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:18.717772    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:18.788364 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:18.788325    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:18.838651 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:18.838619    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:18.918633 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:18.918607    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"
Sep 06 05:13:18.930004 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:18.929972    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:19.004402 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.004341    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4e3b11-22e3-4ea2-b0f7-578004de0116-plugins-config-volume\") pod \"7e4e3b11-22e3-4ea2-b0f7-578004de0116\" (UID: \"7e4e3b11-22e3-4ea2-b0f7-578004de0116\") "
Sep 06 05:13:19.004402 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.004401    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9vlk\" (UniqueName: \"kubernetes.io/projected/7e4e3b11-22e3-4ea2-b0f7-578004de0116-kube-api-access-w9vlk\") pod \"7e4e3b11-22e3-4ea2-b0f7-578004de0116\" (UID: \"7e4e3b11-22e3-4ea2-b0f7-578004de0116\") "
Sep 06 05:13:19.005111 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.005062    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e4e3b11-22e3-4ea2-b0f7-578004de0116-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7e4e3b11-22e3-4ea2-b0f7-578004de0116" (UID: "7e4e3b11-22e3-4ea2-b0f7-578004de0116"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:13:19.010157 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.010113    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e4e3b11-22e3-4ea2-b0f7-578004de0116-kube-api-access-w9vlk" (OuterVolumeSpecName: "kube-api-access-w9vlk") pod "7e4e3b11-22e3-4ea2-b0f7-578004de0116" (UID: "7e4e3b11-22e3-4ea2-b0f7-578004de0116"). InnerVolumeSpecName "kube-api-access-w9vlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:13:19.105504 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.105465    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4e3b11-22e3-4ea2-b0f7-578004de0116-plugins-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:19.105504 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.105499    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9vlk\" (UniqueName: \"kubernetes.io/projected/7e4e3b11-22e3-4ea2-b0f7-578004de0116-kube-api-access-w9vlk\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:19.299527 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.299496    2578 generic.go:356] "Generic (PLEG): container finished" podID="7e4e3b11-22e3-4ea2-b0f7-578004de0116" containerID="60fbe14f49cac34b6f41af768a0606de530f4ef476ca67aaae45a740cba059f9" exitCode=0
Sep 06 05:13:19.299722 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.299532    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"
Sep 06 05:13:19.299722 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.299537    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc" event={"ID":"7e4e3b11-22e3-4ea2-b0f7-578004de0116","Type":"ContainerDied","Data":"60fbe14f49cac34b6f41af768a0606de530f4ef476ca67aaae45a740cba059f9"}
Sep 06 05:13:19.299722 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.299589    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc" event={"ID":"7e4e3b11-22e3-4ea2-b0f7-578004de0116","Type":"ContainerDied","Data":"be3dddaf2aa4347838c7369c007bcda4647e5b180773c570a7cf656d52234198"}
Sep 06 05:13:19.299722 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.299617    2578 scope.go:117] "RemoveContainer" containerID="60fbe14f49cac34b6f41af768a0606de530f4ef476ca67aaae45a740cba059f9"
Sep 06 05:13:19.308645 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.308628    2578 scope.go:117] "RemoveContainer" containerID="60fbe14f49cac34b6f41af768a0606de530f4ef476ca67aaae45a740cba059f9"
Sep 06 05:13:19.308919 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:13:19.308901    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60fbe14f49cac34b6f41af768a0606de530f4ef476ca67aaae45a740cba059f9\": container with ID starting with 60fbe14f49cac34b6f41af768a0606de530f4ef476ca67aaae45a740cba059f9 not found: ID does not exist" containerID="60fbe14f49cac34b6f41af768a0606de530f4ef476ca67aaae45a740cba059f9"
Sep 06 05:13:19.308958 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.308927    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60fbe14f49cac34b6f41af768a0606de530f4ef476ca67aaae45a740cba059f9"} err="failed to get container status \"60fbe14f49cac34b6f41af768a0606de530f4ef476ca67aaae45a740cba059f9\": rpc error: code = NotFound desc = could not find container \"60fbe14f49cac34b6f41af768a0606de530f4ef476ca67aaae45a740cba059f9\": container with ID starting with 60fbe14f49cac34b6f41af768a0606de530f4ef476ca67aaae45a740cba059f9 not found: ID does not exist"
Sep 06 05:13:19.319393 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.319370    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"]
Sep 06 05:13:19.321315 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.321298    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-de5ebc7b-67849486f8-zxlnc"]
Sep 06 05:13:19.575368 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.575269    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:19.675502 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.675470    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:19.774394 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:19.774364    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:21.069644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:21.069587    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:21.153625 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:21.153583    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e4e3b11-22e3-4ea2-b0f7-578004de0116" path="/var/lib/kubelet/pods/7e4e3b11-22e3-4ea2-b0f7-578004de0116/volumes"
Sep 06 05:13:23.157860 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:23.157833    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:23.157860 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:23.157869    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:23.170675 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:23.170641    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:23.333698 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:23.333665    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:24.353303 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:24.353270    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:26.435172 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:26.435095    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"]
Sep 06 05:13:26.472724 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:26.472681    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:27.372618 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:27.372583    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:27.473567 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:27.473522    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:27.582850 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:27.582803    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:27.674571 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:27.674471    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:27.973714 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:27.973638    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"]
Sep 06 05:13:27.990266 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:27.990228    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:28.090932 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:28.090893    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:28.292388 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:28.292293    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:38.523246 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:38.523203    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:38.626461 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:38.626134    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:38.701725 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:38.701688    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:38.750844 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:38.750776    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:39.052946 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:39.052913    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:41.455105 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.455068    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd" podUID="6b4d83b3-b96d-418a-9ca5-422834a553d6" containerName="main" containerID="cri-o://a801059a4541b988c03e68b91142827f6269d4e379a8eb3c7aa45a0402ca6f4f" gracePeriod=45
Sep 06 05:13:41.711748 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.711685    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:13:41.839466 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.839433    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-dshm\") pod \"6b4d83b3-b96d-418a-9ca5-422834a553d6\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") "
Sep 06 05:13:41.839466 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.839470    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b4d83b3-b96d-418a-9ca5-422834a553d6-tls-certs\") pod \"6b4d83b3-b96d-418a-9ca5-422834a553d6\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") "
Sep 06 05:13:41.839721 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.839501    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p58nb\" (UniqueName: \"kubernetes.io/projected/6b4d83b3-b96d-418a-9ca5-422834a553d6-kube-api-access-p58nb\") pod \"6b4d83b3-b96d-418a-9ca5-422834a553d6\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") "
Sep 06 05:13:41.839721 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.839529    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-model-cache\") pod \"6b4d83b3-b96d-418a-9ca5-422834a553d6\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") "
Sep 06 05:13:41.839721 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.839555    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-tmp-dir\") pod \"6b4d83b3-b96d-418a-9ca5-422834a553d6\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") "
Sep 06 05:13:41.839721 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.839613    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-home\") pod \"6b4d83b3-b96d-418a-9ca5-422834a553d6\" (UID: \"6b4d83b3-b96d-418a-9ca5-422834a553d6\") "
Sep 06 05:13:41.839951 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.839872    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6b4d83b3-b96d-418a-9ca5-422834a553d6" (UID: "6b4d83b3-b96d-418a-9ca5-422834a553d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:41.839951 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.839882    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-model-cache" (OuterVolumeSpecName: "model-cache") pod "6b4d83b3-b96d-418a-9ca5-422834a553d6" (UID: "6b4d83b3-b96d-418a-9ca5-422834a553d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:41.839951 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.839897    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-home" (OuterVolumeSpecName: "home") pod "6b4d83b3-b96d-418a-9ca5-422834a553d6" (UID: "6b4d83b3-b96d-418a-9ca5-422834a553d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:41.840159 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.840142    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-home\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:41.840222 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.840159    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-model-cache\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:41.840222 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.840173    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-tmp-dir\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:41.841763 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.841732    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b4d83b3-b96d-418a-9ca5-422834a553d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6b4d83b3-b96d-418a-9ca5-422834a553d6" (UID: "6b4d83b3-b96d-418a-9ca5-422834a553d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:13:41.841871 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.841765    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-dshm" (OuterVolumeSpecName: "dshm") pod "6b4d83b3-b96d-418a-9ca5-422834a553d6" (UID: "6b4d83b3-b96d-418a-9ca5-422834a553d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:41.841871 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.841785    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b4d83b3-b96d-418a-9ca5-422834a553d6-kube-api-access-p58nb" (OuterVolumeSpecName: "kube-api-access-p58nb") pod "6b4d83b3-b96d-418a-9ca5-422834a553d6" (UID: "6b4d83b3-b96d-418a-9ca5-422834a553d6"). InnerVolumeSpecName "kube-api-access-p58nb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:13:41.940984 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.940940    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b4d83b3-b96d-418a-9ca5-422834a553d6-dshm\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:41.940984 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.940977    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b4d83b3-b96d-418a-9ca5-422834a553d6-tls-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:41.940984 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:41.940992    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p58nb\" (UniqueName: \"kubernetes.io/projected/6b4d83b3-b96d-418a-9ca5-422834a553d6-kube-api-access-p58nb\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:42.433362 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:42.433327    2578 generic.go:356] "Generic (PLEG): container finished" podID="6b4d83b3-b96d-418a-9ca5-422834a553d6" containerID="a801059a4541b988c03e68b91142827f6269d4e379a8eb3c7aa45a0402ca6f4f" exitCode=0
Sep 06 05:13:42.433557 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:42.433373    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"
Sep 06 05:13:42.433557 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:42.433411    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd" event={"ID":"6b4d83b3-b96d-418a-9ca5-422834a553d6","Type":"ContainerDied","Data":"a801059a4541b988c03e68b91142827f6269d4e379a8eb3c7aa45a0402ca6f4f"}
Sep 06 05:13:42.433557 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:42.433463    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd" event={"ID":"6b4d83b3-b96d-418a-9ca5-422834a553d6","Type":"ContainerDied","Data":"7d3b1312f526f76268bbb4e6392ad132709853ba1c4ed7254864ce91024f429b"}
Sep 06 05:13:42.433557 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:42.433485    2578 scope.go:117] "RemoveContainer" containerID="a801059a4541b988c03e68b91142827f6269d4e379a8eb3c7aa45a0402ca6f4f"
Sep 06 05:13:42.443190 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:42.443175    2578 scope.go:117] "RemoveContainer" containerID="a801059a4541b988c03e68b91142827f6269d4e379a8eb3c7aa45a0402ca6f4f"
Sep 06 05:13:42.443469 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:13:42.443449    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a801059a4541b988c03e68b91142827f6269d4e379a8eb3c7aa45a0402ca6f4f\": container with ID starting with a801059a4541b988c03e68b91142827f6269d4e379a8eb3c7aa45a0402ca6f4f not found: ID does not exist" containerID="a801059a4541b988c03e68b91142827f6269d4e379a8eb3c7aa45a0402ca6f4f"
Sep 06 05:13:42.443536 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:42.443485    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a801059a4541b988c03e68b91142827f6269d4e379a8eb3c7aa45a0402ca6f4f"} err="failed to get container status \"a801059a4541b988c03e68b91142827f6269d4e379a8eb3c7aa45a0402ca6f4f\": rpc error: code = NotFound desc = could not find container \"a801059a4541b988c03e68b91142827f6269d4e379a8eb3c7aa45a0402ca6f4f\": container with ID starting with a801059a4541b988c03e68b91142827f6269d4e379a8eb3c7aa45a0402ca6f4f not found: ID does not exist"
Sep 06 05:13:42.460141 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:42.455758    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"]
Sep 06 05:13:42.464045 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:42.464004    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-0cd5b238-kserve-847c69d6f6-6m7xd"]
Sep 06 05:13:42.995574 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:42.995533    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46" podUID="6481692e-fb55-465e-8c6c-40400f7f476b" containerName="main" containerID="cri-o://ee708afcaecfe6151a4c892842ee824fcae455ef1af8f134919b292eecb7dbf2" gracePeriod=45
Sep 06 05:13:43.154527 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.154496    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b4d83b3-b96d-418a-9ca5-422834a553d6" path="/var/lib/kubelet/pods/6b4d83b3-b96d-418a-9ca5-422834a553d6/volumes"
Sep 06 05:13:43.254662 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.254604    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:43.350895 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.350862    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s595q\" (UniqueName: \"kubernetes.io/projected/6481692e-fb55-465e-8c6c-40400f7f476b-kube-api-access-s595q\") pod \"6481692e-fb55-465e-8c6c-40400f7f476b\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") "
Sep 06 05:13:43.351106 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.350955    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-dshm\") pod \"6481692e-fb55-465e-8c6c-40400f7f476b\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") "
Sep 06 05:13:43.351106 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.351010    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6481692e-fb55-465e-8c6c-40400f7f476b-tls-certs\") pod \"6481692e-fb55-465e-8c6c-40400f7f476b\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") "
Sep 06 05:13:43.351106 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.351061    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-home\") pod \"6481692e-fb55-465e-8c6c-40400f7f476b\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") "
Sep 06 05:13:43.351289 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.351118    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-model-cache\") pod \"6481692e-fb55-465e-8c6c-40400f7f476b\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") "
Sep 06 05:13:43.351289 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.351175    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-tmp-dir\") pod \"6481692e-fb55-465e-8c6c-40400f7f476b\" (UID: \"6481692e-fb55-465e-8c6c-40400f7f476b\") "
Sep 06 05:13:43.351379 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.351320    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-model-cache" (OuterVolumeSpecName: "model-cache") pod "6481692e-fb55-465e-8c6c-40400f7f476b" (UID: "6481692e-fb55-465e-8c6c-40400f7f476b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:43.351436 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.351355    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-home" (OuterVolumeSpecName: "home") pod "6481692e-fb55-465e-8c6c-40400f7f476b" (UID: "6481692e-fb55-465e-8c6c-40400f7f476b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:43.351436 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.351403    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6481692e-fb55-465e-8c6c-40400f7f476b" (UID: "6481692e-fb55-465e-8c6c-40400f7f476b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:43.351606 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.351587    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-model-cache\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:43.351657 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.351613    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-tmp-dir\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:43.351657 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.351626    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-home\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:43.353259 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.353234    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6481692e-fb55-465e-8c6c-40400f7f476b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6481692e-fb55-465e-8c6c-40400f7f476b" (UID: "6481692e-fb55-465e-8c6c-40400f7f476b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:13:43.353367 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.353290    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-dshm" (OuterVolumeSpecName: "dshm") pod "6481692e-fb55-465e-8c6c-40400f7f476b" (UID: "6481692e-fb55-465e-8c6c-40400f7f476b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:43.353367 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.353295    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6481692e-fb55-465e-8c6c-40400f7f476b-kube-api-access-s595q" (OuterVolumeSpecName: "kube-api-access-s595q") pod "6481692e-fb55-465e-8c6c-40400f7f476b" (UID: "6481692e-fb55-465e-8c6c-40400f7f476b"). InnerVolumeSpecName "kube-api-access-s595q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:13:43.439981 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.439946    2578 generic.go:356] "Generic (PLEG): container finished" podID="6481692e-fb55-465e-8c6c-40400f7f476b" containerID="ee708afcaecfe6151a4c892842ee824fcae455ef1af8f134919b292eecb7dbf2" exitCode=0
Sep 06 05:13:43.440182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.439993    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"
Sep 06 05:13:43.440182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.440046    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46" event={"ID":"6481692e-fb55-465e-8c6c-40400f7f476b","Type":"ContainerDied","Data":"ee708afcaecfe6151a4c892842ee824fcae455ef1af8f134919b292eecb7dbf2"}
Sep 06 05:13:43.440182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.440089    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46" event={"ID":"6481692e-fb55-465e-8c6c-40400f7f476b","Type":"ContainerDied","Data":"2f6cb1b1c65aa6b4d5bea38d8bd7d0c2e8c4d1ca3eb3772a07cb9bb9987a4662"}
Sep 06 05:13:43.440182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.440112    2578 scope.go:117] "RemoveContainer" containerID="ee708afcaecfe6151a4c892842ee824fcae455ef1af8f134919b292eecb7dbf2"
Sep 06 05:13:43.449323 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.449308    2578 scope.go:117] "RemoveContainer" containerID="ee708afcaecfe6151a4c892842ee824fcae455ef1af8f134919b292eecb7dbf2"
Sep 06 05:13:43.449586 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:13:43.449569    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee708afcaecfe6151a4c892842ee824fcae455ef1af8f134919b292eecb7dbf2\": container with ID starting with ee708afcaecfe6151a4c892842ee824fcae455ef1af8f134919b292eecb7dbf2 not found: ID does not exist" containerID="ee708afcaecfe6151a4c892842ee824fcae455ef1af8f134919b292eecb7dbf2"
Sep 06 05:13:43.449652 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.449592    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee708afcaecfe6151a4c892842ee824fcae455ef1af8f134919b292eecb7dbf2"} err="failed to get container status \"ee708afcaecfe6151a4c892842ee824fcae455ef1af8f134919b292eecb7dbf2\": rpc error: code = NotFound desc = could not find container \"ee708afcaecfe6151a4c892842ee824fcae455ef1af8f134919b292eecb7dbf2\": container with ID starting with ee708afcaecfe6151a4c892842ee824fcae455ef1af8f134919b292eecb7dbf2 not found: ID does not exist"
Sep 06 05:13:43.452296 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.452276    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s595q\" (UniqueName: \"kubernetes.io/projected/6481692e-fb55-465e-8c6c-40400f7f476b-kube-api-access-s595q\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:43.452349 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.452300    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6481692e-fb55-465e-8c6c-40400f7f476b-dshm\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:43.452349 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.452310    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6481692e-fb55-465e-8c6c-40400f7f476b-tls-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:43.460102 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.460075    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"]
Sep 06 05:13:43.465783 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:43.465762    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-9afd7f/autores-model-9afd7f-kserve-68894cb59f-59n46"]
Sep 06 05:13:44.749755 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:44.749718    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:44.851326 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:44.851268    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:44.971960 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:44.971928    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"]
Sep 06 05:13:44.972217 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:44.972192    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9" podUID="3b750788-2244-4402-8fc0-f52877293d1b" containerName="istio-proxy" containerID="cri-o://ae970ccfd5072ade9f76bd1474bd69a5139a7e225178e246c6ac147f05b172b6" gracePeriod=30
Sep 06 05:13:44.992340 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:44.992312    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:45.155949 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.155913    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6481692e-fb55-465e-8c6c-40400f7f476b" path="/var/lib/kubelet/pods/6481692e-fb55-465e-8c6c-40400f7f476b/volumes"
Sep 06 05:13:45.413753 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.413658    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:45.674940 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.674865    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"]
Sep 06 05:13:45.675301 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.675288    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b4d83b3-b96d-418a-9ca5-422834a553d6" containerName="main"
Sep 06 05:13:45.675357 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.675304    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b4d83b3-b96d-418a-9ca5-422834a553d6" containerName="main"
Sep 06 05:13:45.675357 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.675316    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6481692e-fb55-465e-8c6c-40400f7f476b" containerName="main"
Sep 06 05:13:45.675357 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.675322    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6481692e-fb55-465e-8c6c-40400f7f476b" containerName="main"
Sep 06 05:13:45.675357 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.675332    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e4e3b11-22e3-4ea2-b0f7-578004de0116" containerName="payload-pre-processing"
Sep 06 05:13:45.675357 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.675338    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e4e3b11-22e3-4ea2-b0f7-578004de0116" containerName="payload-pre-processing"
Sep 06 05:13:45.675510 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.675411    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b4d83b3-b96d-418a-9ca5-422834a553d6" containerName="main"
Sep 06 05:13:45.675510 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.675421    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e4e3b11-22e3-4ea2-b0f7-578004de0116" containerName="payload-pre-processing"
Sep 06 05:13:45.675510 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.675429    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6481692e-fb55-465e-8c6c-40400f7f476b" containerName="main"
Sep 06 05:13:45.679644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.679625    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.681930 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.681908    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-17d595-openshift-default-dockercfg-p54wd\""
Sep 06 05:13:45.690218 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.690193    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"]
Sep 06 05:13:45.702176 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.702151    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:45.775624 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.775585    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-istio-envoy\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.776097 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.775639    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-istio-data\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.776097 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.775716    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85038c9d-1de0-479e-b713-244762e0faa9-istio-podinfo\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.776097 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.775770    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-workload-certs\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.776097 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.775804    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w4mb\" (UniqueName: \"kubernetes.io/projected/85038c9d-1de0-479e-b713-244762e0faa9-kube-api-access-5w4mb\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.776097 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.775862    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85038c9d-1de0-479e-b713-244762e0faa9-istiod-ca-cert\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.776097 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.775921    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-credential-socket\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.776097 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.775998    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-workload-socket\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.776097 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.776088    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85038c9d-1de0-479e-b713-244762e0faa9-istio-token\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.877112 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.877072    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-workload-certs\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.877112 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.877111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w4mb\" (UniqueName: \"kubernetes.io/projected/85038c9d-1de0-479e-b713-244762e0faa9-kube-api-access-5w4mb\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.877336 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.877152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85038c9d-1de0-479e-b713-244762e0faa9-istiod-ca-cert\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.877336 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.877195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-credential-socket\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.877336 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.877254    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-workload-socket\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.877336 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.877294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85038c9d-1de0-479e-b713-244762e0faa9-istio-token\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.877336 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.877323    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-istio-envoy\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.877602 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.877351    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-istio-data\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.877602 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.877397    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85038c9d-1de0-479e-b713-244762e0faa9-istio-podinfo\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.877706 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.877642    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-credential-socket\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.877827 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.877806    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-workload-certs\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.877827 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.877818    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-workload-socket\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.877944 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.877855    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-istio-data\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.877988 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.877937    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85038c9d-1de0-479e-b713-244762e0faa9-istiod-ca-cert\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.879533 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.879514    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-istio-envoy\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.879922 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.879903    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85038c9d-1de0-479e-b713-244762e0faa9-istio-podinfo\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.884302 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.884279    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w4mb\" (UniqueName: \"kubernetes.io/projected/85038c9d-1de0-479e-b713-244762e0faa9-kube-api-access-5w4mb\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.884377 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.884283    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85038c9d-1de0-479e-b713-244762e0faa9-istio-token\") pod \"e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") " pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:45.992223 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:45.992124    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:46.119199 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:46.119163    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:46.271908 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:46.271882    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"]
Sep 06 05:13:46.273771 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:13:46.273740    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85038c9d_1de0_479e_b713_244762e0faa9.slice/crio-aeb0b57a2837a5f2baba77ee0cb5e40f32adabc3a8a805641b312a032292b434 WatchSource:0}: Error finding container aeb0b57a2837a5f2baba77ee0cb5e40f32adabc3a8a805641b312a032292b434: Status 404 returned error can't find the container with id aeb0b57a2837a5f2baba77ee0cb5e40f32adabc3a8a805641b312a032292b434
Sep 06 05:13:46.276185 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:46.276149    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:13:46.276297 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:46.276227    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:13:46.276297 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:46.276276    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:13:46.468943 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:46.468907    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" event={"ID":"85038c9d-1de0-479e-b713-244762e0faa9","Type":"ContainerStarted","Data":"89f35f1828c3fa88581ecb483f33e7e1f17e93d9bcba00f711862e9618d42b4e"}
Sep 06 05:13:46.469136 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:46.468988    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" event={"ID":"85038c9d-1de0-479e-b713-244762e0faa9","Type":"ContainerStarted","Data":"aeb0b57a2837a5f2baba77ee0cb5e40f32adabc3a8a805641b312a032292b434"}
Sep 06 05:13:46.487301 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:46.487223    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" podStartSLOduration=1.487203345 podStartE2EDuration="1.487203345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:13:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:13:46.485466119 +0000 UTC m=+1093.854921371" watchObservedRunningTime="2026-09-06 05:13:46.487203345 +0000 UTC m=+1093.856658591"
Sep 06 05:13:46.992717 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:46.992674    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:47.998642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:47.998610    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:48.356499 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:48.356408    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:48.457460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:48.457419    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:48.479812 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:48.479779    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:48.480729 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:48.480708    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:13:49.499513 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:49.499485    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:49.600761 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:49.600721    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:49.699991 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:49.699959    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:49.963317 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:49.963290    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:50.064786 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.064751    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:50.280346 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.280324    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:13:50.422470 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.422441    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-credential-socket\") pod \"3b750788-2244-4402-8fc0-f52877293d1b\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") "
Sep 06 05:13:50.422651 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.422477    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfmh2\" (UniqueName: \"kubernetes.io/projected/3b750788-2244-4402-8fc0-f52877293d1b-kube-api-access-cfmh2\") pod \"3b750788-2244-4402-8fc0-f52877293d1b\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") "
Sep 06 05:13:50.422651 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.422515    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b750788-2244-4402-8fc0-f52877293d1b-istio-podinfo\") pod \"3b750788-2244-4402-8fc0-f52877293d1b\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") "
Sep 06 05:13:50.422651 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.422541    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-istio-envoy\") pod \"3b750788-2244-4402-8fc0-f52877293d1b\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") "
Sep 06 05:13:50.422651 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.422578    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b750788-2244-4402-8fc0-f52877293d1b-istio-token\") pod \"3b750788-2244-4402-8fc0-f52877293d1b\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") "
Sep 06 05:13:50.422651 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.422649    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-workload-certs\") pod \"3b750788-2244-4402-8fc0-f52877293d1b\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") "
Sep 06 05:13:50.422999 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.422696    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-workload-socket\") pod \"3b750788-2244-4402-8fc0-f52877293d1b\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") "
Sep 06 05:13:50.422999 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.422710    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b750788-2244-4402-8fc0-f52877293d1b" (UID: "3b750788-2244-4402-8fc0-f52877293d1b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:50.422999 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.422742    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b750788-2244-4402-8fc0-f52877293d1b-istiod-ca-cert\") pod \"3b750788-2244-4402-8fc0-f52877293d1b\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") "
Sep 06 05:13:50.422999 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.422777    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-istio-data\") pod \"3b750788-2244-4402-8fc0-f52877293d1b\" (UID: \"3b750788-2244-4402-8fc0-f52877293d1b\") "
Sep 06 05:13:50.422999 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.422891    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b750788-2244-4402-8fc0-f52877293d1b" (UID: "3b750788-2244-4402-8fc0-f52877293d1b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:50.423272 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.423080    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b750788-2244-4402-8fc0-f52877293d1b" (UID: "3b750788-2244-4402-8fc0-f52877293d1b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:50.423272 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.423221    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b750788-2244-4402-8fc0-f52877293d1b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b750788-2244-4402-8fc0-f52877293d1b" (UID: "3b750788-2244-4402-8fc0-f52877293d1b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:13:50.423272 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.423251    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b750788-2244-4402-8fc0-f52877293d1b" (UID: "3b750788-2244-4402-8fc0-f52877293d1b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:50.423432 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.423297    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-credential-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:50.423432 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.423320    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-workload-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:50.423432 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.423334    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-workload-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:50.424815 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.424796    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b750788-2244-4402-8fc0-f52877293d1b-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b750788-2244-4402-8fc0-f52877293d1b" (UID: "3b750788-2244-4402-8fc0-f52877293d1b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:13:50.425118 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.425088    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b750788-2244-4402-8fc0-f52877293d1b-kube-api-access-cfmh2" (OuterVolumeSpecName: "kube-api-access-cfmh2") pod "3b750788-2244-4402-8fc0-f52877293d1b" (UID: "3b750788-2244-4402-8fc0-f52877293d1b"). InnerVolumeSpecName "kube-api-access-cfmh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:13:50.425226 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.425162    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b750788-2244-4402-8fc0-f52877293d1b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b750788-2244-4402-8fc0-f52877293d1b" (UID: "3b750788-2244-4402-8fc0-f52877293d1b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:13:50.425226 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.425206    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b750788-2244-4402-8fc0-f52877293d1b" (UID: "3b750788-2244-4402-8fc0-f52877293d1b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:50.494354 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.494316    2578 generic.go:356] "Generic (PLEG): container finished" podID="3b750788-2244-4402-8fc0-f52877293d1b" containerID="ae970ccfd5072ade9f76bd1474bd69a5139a7e225178e246c6ac147f05b172b6" exitCode=0
Sep 06 05:13:50.494557 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.494360    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"
Sep 06 05:13:50.494557 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.494398    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9" event={"ID":"3b750788-2244-4402-8fc0-f52877293d1b","Type":"ContainerDied","Data":"ae970ccfd5072ade9f76bd1474bd69a5139a7e225178e246c6ac147f05b172b6"}
Sep 06 05:13:50.494557 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.494474    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9" event={"ID":"3b750788-2244-4402-8fc0-f52877293d1b","Type":"ContainerDied","Data":"0bb5148e01ad2249c659e7424c1eeec2df3f38cef2357ed6895175b1b17c82e3"}
Sep 06 05:13:50.494557 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.494516    2578 scope.go:117] "RemoveContainer" containerID="ae970ccfd5072ade9f76bd1474bd69a5139a7e225178e246c6ac147f05b172b6"
Sep 06 05:13:50.504716 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.504695    2578 scope.go:117] "RemoveContainer" containerID="6fe75eb19824bc76437613cd82384375a16e992bff585a7eccc09bebceb5700e"
Sep 06 05:13:50.514329 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.514253    2578 scope.go:117] "RemoveContainer" containerID="ae970ccfd5072ade9f76bd1474bd69a5139a7e225178e246c6ac147f05b172b6"
Sep 06 05:13:50.514667 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:13:50.514645    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae970ccfd5072ade9f76bd1474bd69a5139a7e225178e246c6ac147f05b172b6\": container with ID starting with ae970ccfd5072ade9f76bd1474bd69a5139a7e225178e246c6ac147f05b172b6 not found: ID does not exist" containerID="ae970ccfd5072ade9f76bd1474bd69a5139a7e225178e246c6ac147f05b172b6"
Sep 06 05:13:50.514763 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.514679    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae970ccfd5072ade9f76bd1474bd69a5139a7e225178e246c6ac147f05b172b6"} err="failed to get container status \"ae970ccfd5072ade9f76bd1474bd69a5139a7e225178e246c6ac147f05b172b6\": rpc error: code = NotFound desc = could not find container \"ae970ccfd5072ade9f76bd1474bd69a5139a7e225178e246c6ac147f05b172b6\": container with ID starting with ae970ccfd5072ade9f76bd1474bd69a5139a7e225178e246c6ac147f05b172b6 not found: ID does not exist"
Sep 06 05:13:50.514763 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.514706    2578 scope.go:117] "RemoveContainer" containerID="6fe75eb19824bc76437613cd82384375a16e992bff585a7eccc09bebceb5700e"
Sep 06 05:13:50.515063 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:13:50.515043    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fe75eb19824bc76437613cd82384375a16e992bff585a7eccc09bebceb5700e\": container with ID starting with 6fe75eb19824bc76437613cd82384375a16e992bff585a7eccc09bebceb5700e not found: ID does not exist" containerID="6fe75eb19824bc76437613cd82384375a16e992bff585a7eccc09bebceb5700e"
Sep 06 05:13:50.515147 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.515073    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe75eb19824bc76437613cd82384375a16e992bff585a7eccc09bebceb5700e"} err="failed to get container status \"6fe75eb19824bc76437613cd82384375a16e992bff585a7eccc09bebceb5700e\": rpc error: code = NotFound desc = could not find container \"6fe75eb19824bc76437613cd82384375a16e992bff585a7eccc09bebceb5700e\": container with ID starting with 6fe75eb19824bc76437613cd82384375a16e992bff585a7eccc09bebceb5700e not found: ID does not exist"
Sep 06 05:13:50.516515 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.516494    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"]
Sep 06 05:13:50.525535 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.525507    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfmh2\" (UniqueName: \"kubernetes.io/projected/3b750788-2244-4402-8fc0-f52877293d1b-kube-api-access-cfmh2\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:50.525779 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.525548    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b750788-2244-4402-8fc0-f52877293d1b-istio-podinfo\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:50.525779 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.525563    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-istio-envoy\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:50.525779 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.525577    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b750788-2244-4402-8fc0-f52877293d1b-istio-token\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:50.525779 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.525590    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b750788-2244-4402-8fc0-f52877293d1b-istiod-ca-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:50.525779 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.525607    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b750788-2244-4402-8fc0-f52877293d1b-istio-data\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:13:50.525779 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:50.525716    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-9afd7f-openshift-default-cf4877dbd-48sw9"]
Sep 06 05:13:51.153961 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:51.153930    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b750788-2244-4402-8fc0-f52877293d1b" path="/var/lib/kubelet/pods/3b750788-2244-4402-8fc0-f52877293d1b/volumes"
Sep 06 05:13:53.929676 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:53.929639    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:59.707407 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:59.707343    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:59.759391 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:59.759356    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:59.823990 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:59.823945    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:13:59.924168 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:13:59.924129    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:03.658410 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:03.657689    2578 patch_prober.go:28] interesting pod/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": read tcp 10.133.0.2:54684->10.133.0.59:15021: read: connection reset by peer" start-of-body=
Sep 06 05:14:03.658410 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:03.657768    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" podUID="85038c9d-1de0-479e-b713-244762e0faa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": read tcp 10.133.0.2:54684->10.133.0.59:15021: read: connection reset by peer"
Sep 06 05:14:04.575118 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:04.575086    2578 generic.go:356] "Generic (PLEG): container finished" podID="85038c9d-1de0-479e-b713-244762e0faa9" containerID="89f35f1828c3fa88581ecb483f33e7e1f17e93d9bcba00f711862e9618d42b4e" exitCode=0
Sep 06 05:14:04.575289 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:04.575123    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" event={"ID":"85038c9d-1de0-479e-b713-244762e0faa9","Type":"ContainerDied","Data":"89f35f1828c3fa88581ecb483f33e7e1f17e93d9bcba00f711862e9618d42b4e"}
Sep 06 05:14:04.575554 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:04.575539    2578 scope.go:117] "RemoveContainer" containerID="89f35f1828c3fa88581ecb483f33e7e1f17e93d9bcba00f711862e9618d42b4e"
Sep 06 05:14:04.576555 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:04.576525    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:14:04.576612 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:04.576595    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:14:04.576649 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:04.576626    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:14:05.585001 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:05.582329    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" event={"ID":"85038c9d-1de0-479e-b713-244762e0faa9","Type":"ContainerStarted","Data":"05e0979ecafa591b2e0b656917408e801a9ab4a040286f87683bd0425a11a46b"}
Sep 06 05:14:05.993255 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:05.993216    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:14:05.993466 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:05.993347    2578 patch_prober.go:28] interesting pod/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 06 05:14:05.993466 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:05.993397    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" podUID="85038c9d-1de0-479e-b713-244762e0faa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 06 05:14:06.993519 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:06.993461    2578 patch_prober.go:28] interesting pod/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 06 05:14:06.994053 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:06.993550    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" podUID="85038c9d-1de0-479e-b713-244762e0faa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 06 05:14:07.992577 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:07.992527    2578 patch_prober.go:28] interesting pod/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 06 05:14:07.992774 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:07.992608    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" podUID="85038c9d-1de0-479e-b713-244762e0faa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 06 05:14:08.992619 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:08.992579    2578 patch_prober.go:28] interesting pod/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 06 05:14:08.993115 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:08.992648    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" podUID="85038c9d-1de0-479e-b713-244762e0faa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 06 05:14:09.992735 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:09.992694    2578 patch_prober.go:28] interesting pod/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 06 05:14:09.993310 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:09.993278    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" podUID="85038c9d-1de0-479e-b713-244762e0faa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 06 05:14:10.997112 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:10.997082    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:14:10.997639 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:10.997617    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:14:10.998471 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:10.998450    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:14:13.279883 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:13.279852    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:15.721871 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:15.721832    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:21.383335 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:21.383291    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:21.443298 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:21.443263    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:21.531750 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:21.531711    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:21.629336 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:21.629300    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:22.629437 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.629399    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"]
Sep 06 05:14:22.629921 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.629902    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b750788-2244-4402-8fc0-f52877293d1b" containerName="istio-proxy"
Sep 06 05:14:22.629921 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.629922    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b750788-2244-4402-8fc0-f52877293d1b" containerName="istio-proxy"
Sep 06 05:14:22.630081 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.630031    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b750788-2244-4402-8fc0-f52877293d1b" containerName="istio-proxy"
Sep 06 05:14:22.630081 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.630044    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b750788-2244-4402-8fc0-f52877293d1b" containerName="istio-proxy"
Sep 06 05:14:22.633542 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.633521    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.636689 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.636664    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9035bc\"/\"kube-root-ca.crt\""
Sep 06 05:14:22.636816 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.636667    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9035bc\"/\"test-model-9035bc-kserve-self-signed-certs\""
Sep 06 05:14:22.636816 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.636668    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9035bc\"/\"default-dockercfg-bbfpf\""
Sep 06 05:14:22.636816 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.636710    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9035bc\"/\"openshift-service-ca.crt\""
Sep 06 05:14:22.642150 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.642119    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"]
Sep 06 05:14:22.653110 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.653074    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:22.754916 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.754873    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-home\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.755126 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.754972    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-dshm\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.755126 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.755010    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5pqw\" (UniqueName: \"kubernetes.io/projected/866c379d-59e1-4ca1-85c7-7dad7a3b8710-kube-api-access-j5pqw\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.755126 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.755063    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-tmp-dir\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.755255 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.755141    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-model-cache\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.755255 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.755192    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/866c379d-59e1-4ca1-85c7-7dad7a3b8710-tls-certs\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.856805 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.856770    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-home\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.856805 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.856815    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-dshm\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.857047 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.856834    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5pqw\" (UniqueName: \"kubernetes.io/projected/866c379d-59e1-4ca1-85c7-7dad7a3b8710-kube-api-access-j5pqw\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.857047 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.856855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-tmp-dir\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.857047 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.856898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-model-cache\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.857047 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.856943    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/866c379d-59e1-4ca1-85c7-7dad7a3b8710-tls-certs\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.857306 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.857281    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-home\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.857376 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.857302    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-model-cache\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.857376 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.857359    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-tmp-dir\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.859181 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.859161    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-dshm\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.859452 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.859434    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/866c379d-59e1-4ca1-85c7-7dad7a3b8710-tls-certs\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.864545 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.864522    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5pqw\" (UniqueName: \"kubernetes.io/projected/866c379d-59e1-4ca1-85c7-7dad7a3b8710-kube-api-access-j5pqw\") pod \"test-model-9035bc-kserve-58bcb6b68f-2f8vt\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") " pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:22.944740 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:22.944646    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:23.217757 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:23.217733    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"]
Sep 06 05:14:23.219933 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:14:23.219881    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod866c379d_59e1_4ca1_85c7_7dad7a3b8710.slice/crio-2985a65dce12131c0787d141742f509509a753fb40f667c6311dffb826174016 WatchSource:0}: Error finding container 2985a65dce12131c0787d141742f509509a753fb40f667c6311dffb826174016: Status 404 returned error can't find the container with id 2985a65dce12131c0787d141742f509509a753fb40f667c6311dffb826174016
Sep 06 05:14:23.692087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:23.692051    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt" event={"ID":"866c379d-59e1-4ca1-85c7-7dad7a3b8710","Type":"ContainerStarted","Data":"a2179bb47de75866c56c259d6a640018b7dee4b2dff1b10c3459c00e567cf2e5"}
Sep 06 05:14:23.692087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:23.692093    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt" event={"ID":"866c379d-59e1-4ca1-85c7-7dad7a3b8710","Type":"ContainerStarted","Data":"2985a65dce12131c0787d141742f509509a753fb40f667c6311dffb826174016"}
Sep 06 05:14:23.707672 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:23.707622    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt" podStartSLOduration=1.707606995 podStartE2EDuration="1.707606995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:14:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:14:23.706554434 +0000 UTC m=+1131.076009680" watchObservedRunningTime="2026-09-06 05:14:23.707606995 +0000 UTC m=+1131.077062241"
Sep 06 05:14:32.064518 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:32.064477    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:32.807944 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:32.807912    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:32.945124 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:32.945088    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:32.945297 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:32.945142    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:32.957838 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:32.957809    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:33.758096 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:33.758070    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:14:36.393431 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:36.393394    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:38.554241 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:38.554201    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:38.612656 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:38.612620    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:38.686771 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:38.686738    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:38.786301 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:38.786261    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:38.885938 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:38.885904    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:39.440460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:39.440425    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:39.543059 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:39.542997    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:39.639748 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:39.639713    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:41.046168 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:41.046129    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:44.846069 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.846034    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"]
Sep 06 05:14:44.847217 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.847190    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b750788-2244-4402-8fc0-f52877293d1b" containerName="istio-proxy"
Sep 06 05:14:44.847217 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.847216    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b750788-2244-4402-8fc0-f52877293d1b" containerName="istio-proxy"
Sep 06 05:14:44.851813 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.851794    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.855059 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.855038    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-17d595\"/\"explicit-model-17d595-kserve-self-signed-certs\""
Sep 06 05:14:44.855182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.855040    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-17d595\"/\"openshift-service-ca.crt\""
Sep 06 05:14:44.855182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.855039    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-17d595\"/\"kube-root-ca.crt\""
Sep 06 05:14:44.855182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.855038    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-17d595\"/\"default-dockercfg-7h4bs\""
Sep 06 05:14:44.859509 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.859485    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"]
Sep 06 05:14:44.869747 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.869723    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:44.885684 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.885650    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-model-cache\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.885813 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.885718    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3465f85b-792c-484a-bf5b-0054fd5f7d4b-tls-certs\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.885813 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.885790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-tmp-dir\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.885969 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.885940    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-home\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.886064 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.886044    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2djw\" (UniqueName: \"kubernetes.io/projected/3465f85b-792c-484a-bf5b-0054fd5f7d4b-kube-api-access-z2djw\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.886132 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.886120    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-dshm\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.987444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.987407    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-dshm\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.987625 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.987466    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-model-cache\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.987625 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.987495    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3465f85b-792c-484a-bf5b-0054fd5f7d4b-tls-certs\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.987625 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.987571    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-tmp-dir\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.987625 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.987602    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-home\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.987774 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.987644    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2djw\" (UniqueName: \"kubernetes.io/projected/3465f85b-792c-484a-bf5b-0054fd5f7d4b-kube-api-access-z2djw\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.987943 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.987918    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-model-cache\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.987997 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.987946    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-tmp-dir\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.987997 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.987990    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-home\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.990049 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.989964    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-dshm\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.990539 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.990512    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3465f85b-792c-484a-bf5b-0054fd5f7d4b-tls-certs\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:44.995781 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:44.995751    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2djw\" (UniqueName: \"kubernetes.io/projected/3465f85b-792c-484a-bf5b-0054fd5f7d4b-kube-api-access-z2djw\") pod \"explicit-model-17d595-kserve-79dfb75675-cxbxg\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") " pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:45.163718 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:45.163686    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:45.452219 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:45.452185    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"]
Sep 06 05:14:45.453848 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:14:45.453822    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3465f85b_792c_484a_bf5b_0054fd5f7d4b.slice/crio-0fddf8b1410e2d43d360c315bedf16169bd32499ec91079816a935ee605c4b1c WatchSource:0}: Error finding container 0fddf8b1410e2d43d360c315bedf16169bd32499ec91079816a935ee605c4b1c: Status 404 returned error can't find the container with id 0fddf8b1410e2d43d360c315bedf16169bd32499ec91079816a935ee605c4b1c
Sep 06 05:14:45.814202 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:45.814115    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg" event={"ID":"3465f85b-792c-484a-bf5b-0054fd5f7d4b","Type":"ContainerStarted","Data":"e046c8ddeba4c0ef64cd9d6c2f9850487f74eea99bc7b3bda60884bc862a67c7"}
Sep 06 05:14:45.814202 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:45.814152    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg" event={"ID":"3465f85b-792c-484a-bf5b-0054fd5f7d4b","Type":"ContainerStarted","Data":"0fddf8b1410e2d43d360c315bedf16169bd32499ec91079816a935ee605c4b1c"}
Sep 06 05:14:45.831163 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:45.831119    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg" podStartSLOduration=1.831104162 podStartE2EDuration="1.831104162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:14:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:14:45.828225464 +0000 UTC m=+1153.197680723" watchObservedRunningTime="2026-09-06 05:14:45.831104162 +0000 UTC m=+1153.200559408"
Sep 06 05:14:46.698609 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:46.698573    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:46.796630 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:46.796594    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:47.337744 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:47.337713    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:53.227417 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.227381    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:53.287226 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.287193    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:53.466065 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.466031    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:53.713390 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.713362    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"]
Sep 06 05:14:53.717284 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.717263    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.719493 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.719468    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-241a3769-openshift-default-dockercfg-b6wrk\""
Sep 06 05:14:53.726805 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.726784    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"]
Sep 06 05:14:53.740158 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.740130    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:53.775000 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.774970    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-token\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.775000 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.775001    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d8a61c9-f036-4719-a523-5472c6418b0f-istiod-ca-cert\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.775226 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.775041    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q84cf\" (UniqueName: \"kubernetes.io/projected/8d8a61c9-f036-4719-a523-5472c6418b0f-kube-api-access-q84cf\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.775226 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.775158    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-credential-socket\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.775226 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.775205    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-workload-socket\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.775328 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.775230    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-envoy\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.775328 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.775291    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-podinfo\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.775328 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.775318    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-workload-certs\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.775421 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.775352    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-data\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.876031 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.875992    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-workload-socket\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.876210 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.876099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-envoy\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.876210 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.876147    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-podinfo\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.876210 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.876173    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-workload-certs\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.876210 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.876189    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-data\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.876435 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.876241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-token\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.876435 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.876260    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d8a61c9-f036-4719-a523-5472c6418b0f-istiod-ca-cert\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.876435 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.876285    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q84cf\" (UniqueName: \"kubernetes.io/projected/8d8a61c9-f036-4719-a523-5472c6418b0f-kube-api-access-q84cf\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.876435 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.876388    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-credential-socket\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.876642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.876388    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-workload-socket\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.876642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.876482    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-workload-certs\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.876642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.876579    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-data\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.876811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.876720    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-credential-socket\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.876968 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.876948    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d8a61c9-f036-4719-a523-5472c6418b0f-istiod-ca-cert\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.878331 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.878309    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-envoy\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.878610 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.878592    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-podinfo\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.883588 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.883562    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-token\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:53.883667 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:53.883641    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q84cf\" (UniqueName: \"kubernetes.io/projected/8d8a61c9-f036-4719-a523-5472c6418b0f-kube-api-access-q84cf\") pod \"e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") " pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:54.029369 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:54.029275    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:54.072653 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:54.072614    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:54.305801 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:54.305773    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"]
Sep 06 05:14:54.308156 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:14:54.308126    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d8a61c9_f036_4719_a523_5472c6418b0f.slice/crio-976b5d483a0f1639ddc56e301aaf66de8e687eb87aada97f063c9dbe6e9aa1bb WatchSource:0}: Error finding container 976b5d483a0f1639ddc56e301aaf66de8e687eb87aada97f063c9dbe6e9aa1bb: Status 404 returned error can't find the container with id 976b5d483a0f1639ddc56e301aaf66de8e687eb87aada97f063c9dbe6e9aa1bb
Sep 06 05:14:54.310344 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:54.310310    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:14:54.310460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:54.310391    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:14:54.310460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:54.310443    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:14:54.864527 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:54.864492    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw" event={"ID":"8d8a61c9-f036-4719-a523-5472c6418b0f","Type":"ContainerStarted","Data":"f372c7ceeb78ecc3f899e316c71f67b5738252ebeb54d3d87183d1b443e2f38b"}
Sep 06 05:14:54.864700 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:54.864545    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw" event={"ID":"8d8a61c9-f036-4719-a523-5472c6418b0f","Type":"ContainerStarted","Data":"976b5d483a0f1639ddc56e301aaf66de8e687eb87aada97f063c9dbe6e9aa1bb"}
Sep 06 05:14:54.884118 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:54.884074    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw" podStartSLOduration=1.8840598910000002 podStartE2EDuration="1.884059891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:14:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:14:54.880993339 +0000 UTC m=+1162.250448584" watchObservedRunningTime="2026-09-06 05:14:54.884059891 +0000 UTC m=+1162.253515136"
Sep 06 05:14:55.029845 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:55.029807    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:55.034796 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:55.034755    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:55.164819 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:55.164786    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:55.164983 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:55.164828    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:55.177221 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:55.177193    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:55.869981 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:55.869950    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:55.870826 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:55.870806    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:14:55.881253 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:55.881235    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:14:56.723967 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:56.723936    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:56.819694 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:56.819596    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:57.876000 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:57.875945    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:57.976967 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:57.976932    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:58.173301 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:58.173213    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:58.348335 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:58.348295    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:14:58.440450 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:14:58.440364    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:00.136478 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:00.136446    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs"]
Sep 06 05:15:00.140247 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:00.140228    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs"
Sep 06 05:15:00.142623 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:00.142597    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:15:00.151141 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:00.151112    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs"]
Sep 06 05:15:00.153670 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:00.153648    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs"
Sep 06 05:15:00.423965 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:00.423940    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs"]
Sep 06 05:15:00.425925 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:15:00.425898    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f2e616d_3419_43f5_853b_5f844573fcdb.slice/crio-81e57e17a493b93815a8e05e6bae9d49a3b5e34321746eda93ad76ad99f834c1 WatchSource:0}: Error finding container 81e57e17a493b93815a8e05e6bae9d49a3b5e34321746eda93ad76ad99f834c1: Status 404 returned error can't find the container with id 81e57e17a493b93815a8e05e6bae9d49a3b5e34321746eda93ad76ad99f834c1
Sep 06 05:15:00.427877 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:00.427861    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 05:15:00.900075 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:00.900042    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs" event={"ID":"1f2e616d-3419-43f5-853b-5f844573fcdb","Type":"ContainerStarted","Data":"81e57e17a493b93815a8e05e6bae9d49a3b5e34321746eda93ad76ad99f834c1"}
Sep 06 05:15:03.920782 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:03.920743    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs" event={"ID":"1f2e616d-3419-43f5-853b-5f844573fcdb","Type":"ContainerStarted","Data":"9274f69a65ca6a5365cca7d62a700b6afd6bd0ab76086dab1181f94253ec734d"}
Sep 06 05:15:03.934350 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:03.934302    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs" podStartSLOduration=1.087289552 podStartE2EDuration="3.934287896s" totalImagesPullingTime="2.846998344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:15:03.932311085 +0000 UTC m=+1171.301766332" watchObservedRunningTime="2026-09-06 05:15:03.934287896 +0000 UTC m=+1171.303743141"
Sep 06 05:15:04.934439 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:04.934397    2578 generic.go:356] "Generic (PLEG): container finished" podID="1f2e616d-3419-43f5-853b-5f844573fcdb" containerID="9274f69a65ca6a5365cca7d62a700b6afd6bd0ab76086dab1181f94253ec734d" exitCode=7
Sep 06 05:15:04.934811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:04.934461    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs" event={"ID":"1f2e616d-3419-43f5-853b-5f844573fcdb","Type":"ContainerDied","Data":"9274f69a65ca6a5365cca7d62a700b6afd6bd0ab76086dab1181f94253ec734d"}
Sep 06 05:15:04.934811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:04.934586    2578 scope.go:117] "RemoveContainer" containerID="9274f69a65ca6a5365cca7d62a700b6afd6bd0ab76086dab1181f94253ec734d"
Sep 06 05:15:05.549813 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:05.549735    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"]
Sep 06 05:15:05.567593 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:05.567563    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:05.878682 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:05.878638    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:05.941885 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:05.941851    2578 generic.go:356] "Generic (PLEG): container finished" podID="1f2e616d-3419-43f5-853b-5f844573fcdb" containerID="0de1fd6b69820a1ea4f08c2658e8599ad01d921ca404ef5c772ffaab0f839489" exitCode=7
Sep 06 05:15:05.941885 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:05.941888    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs" event={"ID":"1f2e616d-3419-43f5-853b-5f844573fcdb","Type":"ContainerDied","Data":"0de1fd6b69820a1ea4f08c2658e8599ad01d921ca404ef5c772ffaab0f839489"}
Sep 06 05:15:05.942354 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:05.941933    2578 scope.go:117] "RemoveContainer" containerID="9274f69a65ca6a5365cca7d62a700b6afd6bd0ab76086dab1181f94253ec734d"
Sep 06 05:15:05.942354 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:05.942104    2578 scope.go:117] "RemoveContainer" containerID="0de1fd6b69820a1ea4f08c2658e8599ad01d921ca404ef5c772ffaab0f839489"
Sep 06 05:15:05.942421 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:15:05.942383    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs_odh-ai-gateway-infra(1f2e616d-3419-43f5-853b-5f844573fcdb)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs" podUID="1f2e616d-3419-43f5-853b-5f844573fcdb"
Sep 06 05:15:06.950250 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:06.950221    2578 scope.go:117] "RemoveContainer" containerID="0de1fd6b69820a1ea4f08c2658e8599ad01d921ca404ef5c772ffaab0f839489"
Sep 06 05:15:06.950611 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:15:06.950469    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs_odh-ai-gateway-infra(1f2e616d-3419-43f5-853b-5f844573fcdb)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs" podUID="1f2e616d-3419-43f5-853b-5f844573fcdb"
Sep 06 05:15:16.113040 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:16.112047    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:16.216043 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:16.208865    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:16.309952 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:16.309897    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:16.373271 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:16.373236    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:16.657706 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:16.657608    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:19.147356 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:19.147321    2578 scope.go:117] "RemoveContainer" containerID="0de1fd6b69820a1ea4f08c2658e8599ad01d921ca404ef5c772ffaab0f839489"
Sep 06 05:15:19.786320 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:19.786221    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:20.032080 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.032040    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs" event={"ID":"1f2e616d-3419-43f5-853b-5f844573fcdb","Type":"ContainerStarted","Data":"18214127544cb101040539af8f29e58db527e61e89a5f530c9e718a371b51f75"}
Sep 06 05:15:20.570871 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.570745    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg" podUID="3465f85b-792c-484a-bf5b-0054fd5f7d4b" containerName="main" containerID="cri-o://e046c8ddeba4c0ef64cd9d6c2f9850487f74eea99bc7b3bda60884bc862a67c7" gracePeriod=45
Sep 06 05:15:20.829177 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.829117    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:15:20.871068 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.871035    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-dshm\") pod \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") "
Sep 06 05:15:20.871253 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.871110    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-model-cache\") pod \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") "
Sep 06 05:15:20.871253 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.871206    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-tmp-dir\") pod \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") "
Sep 06 05:15:20.871253 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.871241    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2djw\" (UniqueName: \"kubernetes.io/projected/3465f85b-792c-484a-bf5b-0054fd5f7d4b-kube-api-access-z2djw\") pod \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") "
Sep 06 05:15:20.871428 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.871267    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-home\") pod \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") "
Sep 06 05:15:20.871428 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.871285    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3465f85b-792c-484a-bf5b-0054fd5f7d4b-tls-certs\") pod \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\" (UID: \"3465f85b-792c-484a-bf5b-0054fd5f7d4b\") "
Sep 06 05:15:20.871428 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.871368    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-model-cache" (OuterVolumeSpecName: "model-cache") pod "3465f85b-792c-484a-bf5b-0054fd5f7d4b" (UID: "3465f85b-792c-484a-bf5b-0054fd5f7d4b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:20.871590 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.871454    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3465f85b-792c-484a-bf5b-0054fd5f7d4b" (UID: "3465f85b-792c-484a-bf5b-0054fd5f7d4b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:20.871679 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.871658    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-home" (OuterVolumeSpecName: "home") pod "3465f85b-792c-484a-bf5b-0054fd5f7d4b" (UID: "3465f85b-792c-484a-bf5b-0054fd5f7d4b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:20.871748 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.871714    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-model-cache\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:20.871748 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.871734    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-tmp-dir\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:20.873480 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.873459    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-dshm" (OuterVolumeSpecName: "dshm") pod "3465f85b-792c-484a-bf5b-0054fd5f7d4b" (UID: "3465f85b-792c-484a-bf5b-0054fd5f7d4b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:20.873877 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.873851    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3465f85b-792c-484a-bf5b-0054fd5f7d4b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3465f85b-792c-484a-bf5b-0054fd5f7d4b" (UID: "3465f85b-792c-484a-bf5b-0054fd5f7d4b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:15:20.873953 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.873854    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3465f85b-792c-484a-bf5b-0054fd5f7d4b-kube-api-access-z2djw" (OuterVolumeSpecName: "kube-api-access-z2djw") pod "3465f85b-792c-484a-bf5b-0054fd5f7d4b" (UID: "3465f85b-792c-484a-bf5b-0054fd5f7d4b"). InnerVolumeSpecName "kube-api-access-z2djw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:15:20.972788 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.972754    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2djw\" (UniqueName: \"kubernetes.io/projected/3465f85b-792c-484a-bf5b-0054fd5f7d4b-kube-api-access-z2djw\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:20.972788 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.972782    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-home\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:20.972984 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.972796    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3465f85b-792c-484a-bf5b-0054fd5f7d4b-tls-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:20.972984 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:20.972809    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3465f85b-792c-484a-bf5b-0054fd5f7d4b-dshm\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:21.038724 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.038690    2578 generic.go:356] "Generic (PLEG): container finished" podID="8d8a61c9-f036-4719-a523-5472c6418b0f" containerID="f372c7ceeb78ecc3f899e316c71f67b5738252ebeb54d3d87183d1b443e2f38b" exitCode=0
Sep 06 05:15:21.038890 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.038766    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw" event={"ID":"8d8a61c9-f036-4719-a523-5472c6418b0f","Type":"ContainerDied","Data":"f372c7ceeb78ecc3f899e316c71f67b5738252ebeb54d3d87183d1b443e2f38b"}
Sep 06 05:15:21.039247 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.039225    2578 scope.go:117] "RemoveContainer" containerID="f372c7ceeb78ecc3f899e316c71f67b5738252ebeb54d3d87183d1b443e2f38b"
Sep 06 05:15:21.040114 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.040090    2578 generic.go:356] "Generic (PLEG): container finished" podID="3465f85b-792c-484a-bf5b-0054fd5f7d4b" containerID="e046c8ddeba4c0ef64cd9d6c2f9850487f74eea99bc7b3bda60884bc862a67c7" exitCode=0
Sep 06 05:15:21.040227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.040163    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"
Sep 06 05:15:21.040227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.040199    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg" event={"ID":"3465f85b-792c-484a-bf5b-0054fd5f7d4b","Type":"ContainerDied","Data":"e046c8ddeba4c0ef64cd9d6c2f9850487f74eea99bc7b3bda60884bc862a67c7"}
Sep 06 05:15:21.040347 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.040244    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg" event={"ID":"3465f85b-792c-484a-bf5b-0054fd5f7d4b","Type":"ContainerDied","Data":"0fddf8b1410e2d43d360c315bedf16169bd32499ec91079816a935ee605c4b1c"}
Sep 06 05:15:21.040347 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.040267    2578 scope.go:117] "RemoveContainer" containerID="e046c8ddeba4c0ef64cd9d6c2f9850487f74eea99bc7b3bda60884bc862a67c7"
Sep 06 05:15:21.040455 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.040419    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:15:21.040522 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.040504    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:15:21.040576 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.040560    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:15:21.042673 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.042647    2578 generic.go:356] "Generic (PLEG): container finished" podID="1f2e616d-3419-43f5-853b-5f844573fcdb" containerID="18214127544cb101040539af8f29e58db527e61e89a5f530c9e718a371b51f75" exitCode=0
Sep 06 05:15:21.042777 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.042678    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs" event={"ID":"1f2e616d-3419-43f5-853b-5f844573fcdb","Type":"ContainerDied","Data":"18214127544cb101040539af8f29e58db527e61e89a5f530c9e718a371b51f75"}
Sep 06 05:15:21.052709 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.052690    2578 scope.go:117] "RemoveContainer" containerID="e046c8ddeba4c0ef64cd9d6c2f9850487f74eea99bc7b3bda60884bc862a67c7"
Sep 06 05:15:21.052998 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:15:21.052978    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e046c8ddeba4c0ef64cd9d6c2f9850487f74eea99bc7b3bda60884bc862a67c7\": container with ID starting with e046c8ddeba4c0ef64cd9d6c2f9850487f74eea99bc7b3bda60884bc862a67c7 not found: ID does not exist" containerID="e046c8ddeba4c0ef64cd9d6c2f9850487f74eea99bc7b3bda60884bc862a67c7"
Sep 06 05:15:21.053095 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.053006    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e046c8ddeba4c0ef64cd9d6c2f9850487f74eea99bc7b3bda60884bc862a67c7"} err="failed to get container status \"e046c8ddeba4c0ef64cd9d6c2f9850487f74eea99bc7b3bda60884bc862a67c7\": rpc error: code = NotFound desc = could not find container \"e046c8ddeba4c0ef64cd9d6c2f9850487f74eea99bc7b3bda60884bc862a67c7\": container with ID starting with e046c8ddeba4c0ef64cd9d6c2f9850487f74eea99bc7b3bda60884bc862a67c7 not found: ID does not exist"
Sep 06 05:15:21.053095 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.053037    2578 scope.go:117] "RemoveContainer" containerID="0de1fd6b69820a1ea4f08c2658e8599ad01d921ca404ef5c772ffaab0f839489"
Sep 06 05:15:21.054452 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.054423    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs"]
Sep 06 05:15:21.086367 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.086335    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"]
Sep 06 05:15:21.089610 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.089579    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-17d595/explicit-model-17d595-kserve-79dfb75675-cxbxg"]
Sep 06 05:15:21.156377 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.156343    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3465f85b-792c-484a-bf5b-0054fd5f7d4b" path="/var/lib/kubelet/pods/3465f85b-792c-484a-bf5b-0054fd5f7d4b/volumes"
Sep 06 05:15:21.738283 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:21.738243    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:22.050036 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:22.049926    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw" event={"ID":"8d8a61c9-f036-4719-a523-5472c6418b0f","Type":"ContainerStarted","Data":"de23e00d2303fdb9cf19868efb73660b44c9a5247d17fd520cde5cba0b319c2c"}
Sep 06 05:15:22.225973 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:22.225908    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs"
Sep 06 05:15:22.392874 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:22.392813    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:22.495261 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:22.495178    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:22.794645 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:22.794555    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:23.030130 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.030095    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:15:23.031368 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.031344    2578 patch_prober.go:28] interesting pod/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 06 05:15:23.031427 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.031393    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw" podUID="8d8a61c9-f036-4719-a523-5472c6418b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 06 05:15:23.044467 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.044437    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:23.059497 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.059424    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs"
Sep 06 05:15:23.059497 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.059453    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs" event={"ID":"1f2e616d-3419-43f5-853b-5f844573fcdb","Type":"ContainerDied","Data":"81e57e17a493b93815a8e05e6bae9d49a3b5e34321746eda93ad76ad99f834c1"}
Sep 06 05:15:23.059497 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.059497    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81e57e17a493b93815a8e05e6bae9d49a3b5e34321746eda93ad76ad99f834c1"
Sep 06 05:15:23.085152 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.085120    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs"]
Sep 06 05:15:23.089062 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.089009    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-241a3769-29811195-pzccs"]
Sep 06 05:15:23.145030 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.144993    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:23.153538 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.153512    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f2e616d-3419-43f5-853b-5f844573fcdb" path="/var/lib/kubelet/pods/1f2e616d-3419-43f5-853b-5f844573fcdb/volumes"
Sep 06 05:15:23.268722 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.268680    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"]
Sep 06 05:15:23.268964 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.268939    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" podUID="85038c9d-1de0-479e-b713-244762e0faa9" containerName="istio-proxy" containerID="cri-o://05e0979ecafa591b2e0b656917408e801a9ab4a040286f87683bd0425a11a46b" gracePeriod=30
Sep 06 05:15:23.302799 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.302562    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:23.703760 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.703703    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:23.994551 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:23.994417    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:24.030522 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:24.030488    2578 patch_prober.go:28] interesting pod/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 06 05:15:24.030699 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:24.030544    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw" podUID="8d8a61c9-f036-4719-a523-5472c6418b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 06 05:15:24.327262 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:24.327186    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:25.029958 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:25.029916    2578 patch_prober.go:28] interesting pod/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 06 05:15:25.030372 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:25.029975    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw" podUID="8d8a61c9-f036-4719-a523-5472c6418b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 06 05:15:25.870726 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:25.870690    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:15:25.877414 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:25.877387    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:26.005480 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:26.005432    2578 patch_prober.go:28] interesting pod/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 05:15:26.005671 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:26.005528    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" podUID="85038c9d-1de0-479e-b713-244762e0faa9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 05:15:26.030434 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:26.030397    2578 patch_prober.go:28] interesting pod/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 06 05:15:26.030807 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:26.030479    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw" podUID="8d8a61c9-f036-4719-a523-5472c6418b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 06 05:15:27.030493 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:27.030457    2578 patch_prober.go:28] interesting pod/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 06 05:15:27.030884 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:27.030517    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw" podUID="8d8a61c9-f036-4719-a523-5472c6418b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 06 05:15:27.055608 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:27.055575    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:28.034689 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.034659    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:15:28.035770 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.035744    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:15:28.375774 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.375740    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:28.475712 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.475678    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:28.522466 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.522441    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:15:28.583859 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.583773    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-istio-data\") pod \"85038c9d-1de0-479e-b713-244762e0faa9\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") "
Sep 06 05:15:28.583859 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.583817    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-workload-socket\") pod \"85038c9d-1de0-479e-b713-244762e0faa9\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") "
Sep 06 05:15:28.583859 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.583846    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85038c9d-1de0-479e-b713-244762e0faa9-istiod-ca-cert\") pod \"85038c9d-1de0-479e-b713-244762e0faa9\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") "
Sep 06 05:15:28.584134 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.583873    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-workload-certs\") pod \"85038c9d-1de0-479e-b713-244762e0faa9\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") "
Sep 06 05:15:28.584134 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.583902    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-istio-envoy\") pod \"85038c9d-1de0-479e-b713-244762e0faa9\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") "
Sep 06 05:15:28.584134 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.583931    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85038c9d-1de0-479e-b713-244762e0faa9-istio-podinfo\") pod \"85038c9d-1de0-479e-b713-244762e0faa9\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") "
Sep 06 05:15:28.584134 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.583993    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5w4mb\" (UniqueName: \"kubernetes.io/projected/85038c9d-1de0-479e-b713-244762e0faa9-kube-api-access-5w4mb\") pod \"85038c9d-1de0-479e-b713-244762e0faa9\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") "
Sep 06 05:15:28.584134 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.584040    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85038c9d-1de0-479e-b713-244762e0faa9-istio-token\") pod \"85038c9d-1de0-479e-b713-244762e0faa9\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") "
Sep 06 05:15:28.584134 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.584090    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-credential-socket\") pod \"85038c9d-1de0-479e-b713-244762e0faa9\" (UID: \"85038c9d-1de0-479e-b713-244762e0faa9\") "
Sep 06 05:15:28.584421 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.584143    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-istio-data" (OuterVolumeSpecName: "istio-data") pod "85038c9d-1de0-479e-b713-244762e0faa9" (UID: "85038c9d-1de0-479e-b713-244762e0faa9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:28.584421 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.584139    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "85038c9d-1de0-479e-b713-244762e0faa9" (UID: "85038c9d-1de0-479e-b713-244762e0faa9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:28.584421 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.584221    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "85038c9d-1de0-479e-b713-244762e0faa9" (UID: "85038c9d-1de0-479e-b713-244762e0faa9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:28.584421 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.584265    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85038c9d-1de0-479e-b713-244762e0faa9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "85038c9d-1de0-479e-b713-244762e0faa9" (UID: "85038c9d-1de0-479e-b713-244762e0faa9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:15:28.584555 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.584449    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "85038c9d-1de0-479e-b713-244762e0faa9" (UID: "85038c9d-1de0-479e-b713-244762e0faa9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:28.584694 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.584673    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-workload-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:28.584774 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.584694    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-credential-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:28.584774 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.584703    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-istio-data\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:28.584774 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.584711    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-workload-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:28.584774 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.584719    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85038c9d-1de0-479e-b713-244762e0faa9-istiod-ca-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:28.586437 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.586395    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85038c9d-1de0-479e-b713-244762e0faa9-kube-api-access-5w4mb" (OuterVolumeSpecName: "kube-api-access-5w4mb") pod "85038c9d-1de0-479e-b713-244762e0faa9" (UID: "85038c9d-1de0-479e-b713-244762e0faa9"). InnerVolumeSpecName "kube-api-access-5w4mb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:15:28.586437 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.586405    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85038c9d-1de0-479e-b713-244762e0faa9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "85038c9d-1de0-479e-b713-244762e0faa9" (UID: "85038c9d-1de0-479e-b713-244762e0faa9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:15:28.586657 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.586637    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "85038c9d-1de0-479e-b713-244762e0faa9" (UID: "85038c9d-1de0-479e-b713-244762e0faa9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:28.586657 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.586647    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85038c9d-1de0-479e-b713-244762e0faa9-istio-token" (OuterVolumeSpecName: "istio-token") pod "85038c9d-1de0-479e-b713-244762e0faa9" (UID: "85038c9d-1de0-479e-b713-244762e0faa9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:15:28.686201 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.686165    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85038c9d-1de0-479e-b713-244762e0faa9-istio-envoy\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:28.686201 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.686199    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85038c9d-1de0-479e-b713-244762e0faa9-istio-podinfo\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:28.686201 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.686209    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w4mb\" (UniqueName: \"kubernetes.io/projected/85038c9d-1de0-479e-b713-244762e0faa9-kube-api-access-5w4mb\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:28.686434 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:28.686219    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85038c9d-1de0-479e-b713-244762e0faa9-istio-token\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:29.100642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:29.100603    2578 generic.go:356] "Generic (PLEG): container finished" podID="85038c9d-1de0-479e-b713-244762e0faa9" containerID="05e0979ecafa591b2e0b656917408e801a9ab4a040286f87683bd0425a11a46b" exitCode=0
Sep 06 05:15:29.100642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:29.100644    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" event={"ID":"85038c9d-1de0-479e-b713-244762e0faa9","Type":"ContainerDied","Data":"05e0979ecafa591b2e0b656917408e801a9ab4a040286f87683bd0425a11a46b"}
Sep 06 05:15:29.101209 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:29.100652    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"
Sep 06 05:15:29.101209 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:29.100686    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj" event={"ID":"85038c9d-1de0-479e-b713-244762e0faa9","Type":"ContainerDied","Data":"aeb0b57a2837a5f2baba77ee0cb5e40f32adabc3a8a805641b312a032292b434"}
Sep 06 05:15:29.101209 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:29.100709    2578 scope.go:117] "RemoveContainer" containerID="05e0979ecafa591b2e0b656917408e801a9ab4a040286f87683bd0425a11a46b"
Sep 06 05:15:29.111212 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:29.111149    2578 scope.go:117] "RemoveContainer" containerID="89f35f1828c3fa88581ecb483f33e7e1f17e93d9bcba00f711862e9618d42b4e"
Sep 06 05:15:29.120417 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:29.120399    2578 scope.go:117] "RemoveContainer" containerID="05e0979ecafa591b2e0b656917408e801a9ab4a040286f87683bd0425a11a46b"
Sep 06 05:15:29.120816 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:15:29.120792    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05e0979ecafa591b2e0b656917408e801a9ab4a040286f87683bd0425a11a46b\": container with ID starting with 05e0979ecafa591b2e0b656917408e801a9ab4a040286f87683bd0425a11a46b not found: ID does not exist" containerID="05e0979ecafa591b2e0b656917408e801a9ab4a040286f87683bd0425a11a46b"
Sep 06 05:15:29.120933 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:29.120826    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e0979ecafa591b2e0b656917408e801a9ab4a040286f87683bd0425a11a46b"} err="failed to get container status \"05e0979ecafa591b2e0b656917408e801a9ab4a040286f87683bd0425a11a46b\": rpc error: code = NotFound desc = could not find container \"05e0979ecafa591b2e0b656917408e801a9ab4a040286f87683bd0425a11a46b\": container with ID starting with 05e0979ecafa591b2e0b656917408e801a9ab4a040286f87683bd0425a11a46b not found: ID does not exist"
Sep 06 05:15:29.120933 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:29.120850    2578 scope.go:117] "RemoveContainer" containerID="89f35f1828c3fa88581ecb483f33e7e1f17e93d9bcba00f711862e9618d42b4e"
Sep 06 05:15:29.121213 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:15:29.121190    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89f35f1828c3fa88581ecb483f33e7e1f17e93d9bcba00f711862e9618d42b4e\": container with ID starting with 89f35f1828c3fa88581ecb483f33e7e1f17e93d9bcba00f711862e9618d42b4e not found: ID does not exist" containerID="89f35f1828c3fa88581ecb483f33e7e1f17e93d9bcba00f711862e9618d42b4e"
Sep 06 05:15:29.121295 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:29.121222    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89f35f1828c3fa88581ecb483f33e7e1f17e93d9bcba00f711862e9618d42b4e"} err="failed to get container status \"89f35f1828c3fa88581ecb483f33e7e1f17e93d9bcba00f711862e9618d42b4e\": rpc error: code = NotFound desc = could not find container \"89f35f1828c3fa88581ecb483f33e7e1f17e93d9bcba00f711862e9618d42b4e\": container with ID starting with 89f35f1828c3fa88581ecb483f33e7e1f17e93d9bcba00f711862e9618d42b4e not found: ID does not exist"
Sep 06 05:15:29.123792 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:29.123770    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"]
Sep 06 05:15:29.125742 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:29.125723    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-17d595-openshift-default-db7bccddd-rtmbj"]
Sep 06 05:15:29.154529 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:29.154504    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85038c9d-1de0-479e-b713-244762e0faa9" path="/var/lib/kubelet/pods/85038c9d-1de0-479e-b713-244762e0faa9/volumes"
Sep 06 05:15:32.740760 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.740724    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"]
Sep 06 05:15:32.741170 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741155    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3465f85b-792c-484a-bf5b-0054fd5f7d4b" containerName="main"
Sep 06 05:15:32.741170 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741170    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3465f85b-792c-484a-bf5b-0054fd5f7d4b" containerName="main"
Sep 06 05:15:32.741252 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741180    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85038c9d-1de0-479e-b713-244762e0faa9" containerName="istio-proxy"
Sep 06 05:15:32.741252 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741186    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="85038c9d-1de0-479e-b713-244762e0faa9" containerName="istio-proxy"
Sep 06 05:15:32.741252 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741197    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f2e616d-3419-43f5-853b-5f844573fcdb" containerName="cleanup"
Sep 06 05:15:32.741252 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741202    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2e616d-3419-43f5-853b-5f844573fcdb" containerName="cleanup"
Sep 06 05:15:32.741252 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741208    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f2e616d-3419-43f5-853b-5f844573fcdb" containerName="cleanup"
Sep 06 05:15:32.741252 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741213    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2e616d-3419-43f5-853b-5f844573fcdb" containerName="cleanup"
Sep 06 05:15:32.741252 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741221    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85038c9d-1de0-479e-b713-244762e0faa9" containerName="istio-proxy"
Sep 06 05:15:32.741252 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741227    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="85038c9d-1de0-479e-b713-244762e0faa9" containerName="istio-proxy"
Sep 06 05:15:32.741252 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741234    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f2e616d-3419-43f5-853b-5f844573fcdb" containerName="cleanup"
Sep 06 05:15:32.741252 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741239    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2e616d-3419-43f5-853b-5f844573fcdb" containerName="cleanup"
Sep 06 05:15:32.741569 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741299    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="85038c9d-1de0-479e-b713-244762e0faa9" containerName="istio-proxy"
Sep 06 05:15:32.741569 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741307    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f2e616d-3419-43f5-853b-5f844573fcdb" containerName="cleanup"
Sep 06 05:15:32.741569 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741314    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="85038c9d-1de0-479e-b713-244762e0faa9" containerName="istio-proxy"
Sep 06 05:15:32.741569 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741321    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f2e616d-3419-43f5-853b-5f844573fcdb" containerName="cleanup"
Sep 06 05:15:32.741569 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.741327    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3465f85b-792c-484a-bf5b-0054fd5f7d4b" containerName="main"
Sep 06 05:15:32.744417 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.744401    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.746382 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.746360    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-c53189\"/\"default-dockercfg-7xg5b\""
Sep 06 05:15:32.747380 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.747355    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-c53189\"/\"noten-model-c53189-kserve-self-signed-certs\""
Sep 06 05:15:32.747478 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.747383    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-c53189\"/\"openshift-service-ca.crt\""
Sep 06 05:15:32.747478 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.747361    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-c53189\"/\"kube-root-ca.crt\""
Sep 06 05:15:32.751393 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.751370    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"]
Sep 06 05:15:32.828491 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.828458    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-dshm\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.828665 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.828583    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c87rd\" (UniqueName: \"kubernetes.io/projected/4673dad9-60b4-4370-a0b5-d24616630922-kube-api-access-c87rd\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.828665 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.828617    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-model-cache\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.828665 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.828642    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-home\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.828832 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.828683    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-tmp-dir\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.828832 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.828741    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4673dad9-60b4-4370-a0b5-d24616630922-tls-certs\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.835674 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.835626    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:32.930224 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.930180    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-home\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.930924 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.930901    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-tmp-dir\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.931123 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.931106    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4673dad9-60b4-4370-a0b5-d24616630922-tls-certs\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.931337 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.931309    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-dshm\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.931515 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.931449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c87rd\" (UniqueName: \"kubernetes.io/projected/4673dad9-60b4-4370-a0b5-d24616630922-kube-api-access-c87rd\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.931612 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.931530    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-model-cache\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.931954 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.931918    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-model-cache\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.932087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.930843    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-home\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.932367 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.932339    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-tmp-dir\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.934811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.934786    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-dshm\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.935036 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.935004    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4673dad9-60b4-4370-a0b5-d24616630922-tls-certs\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:32.939878 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:32.939842    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c87rd\" (UniqueName: \"kubernetes.io/projected/4673dad9-60b4-4370-a0b5-d24616630922-kube-api-access-c87rd\") pod \"noten-model-c53189-kserve-f76657795-tb8qb\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") " pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:33.058345 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:33.058280    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-c53189\"/\"default-dockercfg-7xg5b\""
Sep 06 05:15:33.066268 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:33.066250    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:33.351169 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:33.351140    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"]
Sep 06 05:15:33.353167 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:15:33.353139    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4673dad9_60b4_4370_a0b5_d24616630922.slice/crio-33bda18bfea859f1c7c2b0b81a636107f76d804bf8a1cc26aee202b0a9666b64 WatchSource:0}: Error finding container 33bda18bfea859f1c7c2b0b81a636107f76d804bf8a1cc26aee202b0a9666b64: Status 404 returned error can't find the container with id 33bda18bfea859f1c7c2b0b81a636107f76d804bf8a1cc26aee202b0a9666b64
Sep 06 05:15:34.139046 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:34.138997    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb" event={"ID":"4673dad9-60b4-4370-a0b5-d24616630922","Type":"ContainerStarted","Data":"cb734ac621e5f09ea544619aa073af26a5d2b8e2fa627fe93405e92e5d17bd38"}
Sep 06 05:15:34.139471 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:34.139057    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb" event={"ID":"4673dad9-60b4-4370-a0b5-d24616630922","Type":"ContainerStarted","Data":"33bda18bfea859f1c7c2b0b81a636107f76d804bf8a1cc26aee202b0a9666b64"}
Sep 06 05:15:34.152821 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:34.152769    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb" podStartSLOduration=2.152749307 podStartE2EDuration="2.152749307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:15:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:15:34.152595698 +0000 UTC m=+1201.522050944" watchObservedRunningTime="2026-09-06 05:15:34.152749307 +0000 UTC m=+1201.522204552"
Sep 06 05:15:39.124160 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:39.124122    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:39.231758 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:39.231722    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:39.290694 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:39.290659    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:39.630561 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:39.630530    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:43.067281 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:43.067240    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:43.067758 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:43.067296    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:43.079428 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:43.079404    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:43.201598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:43.201570    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:15:45.306402 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:45.306366    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:45.407258 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:45.407225    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:45.529102 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:45.529064    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"]
Sep 06 05:15:45.529395 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:45.529364    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw" podUID="8d8a61c9-f036-4719-a523-5472c6418b0f" containerName="istio-proxy" containerID="cri-o://de23e00d2303fdb9cf19868efb73660b44c9a5247d17fd520cde5cba0b319c2c" gracePeriod=30
Sep 06 05:15:45.542683 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:45.542653    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:45.859044 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:45.858988    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:45.959801 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:45.959768    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:46.077755 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:46.077721    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:50.793634 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.793567    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:15:50.914029 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.913985    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-podinfo\") pod \"8d8a61c9-f036-4719-a523-5472c6418b0f\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") "
Sep 06 05:15:50.914219 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.914061    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q84cf\" (UniqueName: \"kubernetes.io/projected/8d8a61c9-f036-4719-a523-5472c6418b0f-kube-api-access-q84cf\") pod \"8d8a61c9-f036-4719-a523-5472c6418b0f\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") "
Sep 06 05:15:50.914219 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.914114    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-credential-socket\") pod \"8d8a61c9-f036-4719-a523-5472c6418b0f\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") "
Sep 06 05:15:50.914219 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.914192    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-envoy\") pod \"8d8a61c9-f036-4719-a523-5472c6418b0f\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") "
Sep 06 05:15:50.914410 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.914228    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-workload-certs\") pod \"8d8a61c9-f036-4719-a523-5472c6418b0f\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") "
Sep 06 05:15:50.914410 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.914271    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d8a61c9-f036-4719-a523-5472c6418b0f-istiod-ca-cert\") pod \"8d8a61c9-f036-4719-a523-5472c6418b0f\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") "
Sep 06 05:15:50.914410 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.914300    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-token\") pod \"8d8a61c9-f036-4719-a523-5472c6418b0f\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") "
Sep 06 05:15:50.914410 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.914356    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-workload-socket\") pod \"8d8a61c9-f036-4719-a523-5472c6418b0f\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") "
Sep 06 05:15:50.914410 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.914390    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-data\") pod \"8d8a61c9-f036-4719-a523-5472c6418b0f\" (UID: \"8d8a61c9-f036-4719-a523-5472c6418b0f\") "
Sep 06 05:15:50.914691 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.914636    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d8a61c9-f036-4719-a523-5472c6418b0f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8d8a61c9-f036-4719-a523-5472c6418b0f" (UID: "8d8a61c9-f036-4719-a523-5472c6418b0f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:15:50.914820 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.914801    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d8a61c9-f036-4719-a523-5472c6418b0f-istiod-ca-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:50.914820 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.914809    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-data" (OuterVolumeSpecName: "istio-data") pod "8d8a61c9-f036-4719-a523-5472c6418b0f" (UID: "8d8a61c9-f036-4719-a523-5472c6418b0f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:50.914988 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.914965    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8d8a61c9-f036-4719-a523-5472c6418b0f" (UID: "8d8a61c9-f036-4719-a523-5472c6418b0f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:50.915167 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.914992    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8d8a61c9-f036-4719-a523-5472c6418b0f" (UID: "8d8a61c9-f036-4719-a523-5472c6418b0f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:50.915263 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.915178    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8d8a61c9-f036-4719-a523-5472c6418b0f" (UID: "8d8a61c9-f036-4719-a523-5472c6418b0f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:50.917340 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.917309    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d8a61c9-f036-4719-a523-5472c6418b0f-kube-api-access-q84cf" (OuterVolumeSpecName: "kube-api-access-q84cf") pod "8d8a61c9-f036-4719-a523-5472c6418b0f" (UID: "8d8a61c9-f036-4719-a523-5472c6418b0f"). InnerVolumeSpecName "kube-api-access-q84cf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:15:50.917458 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.917391    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8d8a61c9-f036-4719-a523-5472c6418b0f" (UID: "8d8a61c9-f036-4719-a523-5472c6418b0f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:50.917637 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.917602    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8d8a61c9-f036-4719-a523-5472c6418b0f" (UID: "8d8a61c9-f036-4719-a523-5472c6418b0f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:15:50.918038 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.917983    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-token" (OuterVolumeSpecName: "istio-token") pod "8d8a61c9-f036-4719-a523-5472c6418b0f" (UID: "8d8a61c9-f036-4719-a523-5472c6418b0f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:15:50.940335 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:50.940286    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:51.015689 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.015635    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-credential-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:51.015689 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.015683    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-envoy\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:51.015689 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.015696    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-workload-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:51.016030 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.015707    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-token\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:51.016030 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.015718    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-workload-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:51.016030 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.015728    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-data\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:51.016030 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.015739    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d8a61c9-f036-4719-a523-5472c6418b0f-istio-podinfo\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:51.016030 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.015750    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q84cf\" (UniqueName: \"kubernetes.io/projected/8d8a61c9-f036-4719-a523-5472c6418b0f-kube-api-access-q84cf\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:15:51.232726 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.232692    2578 generic.go:356] "Generic (PLEG): container finished" podID="8d8a61c9-f036-4719-a523-5472c6418b0f" containerID="de23e00d2303fdb9cf19868efb73660b44c9a5247d17fd520cde5cba0b319c2c" exitCode=0
Sep 06 05:15:51.232924 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.232757    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"
Sep 06 05:15:51.232924 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.232788    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw" event={"ID":"8d8a61c9-f036-4719-a523-5472c6418b0f","Type":"ContainerDied","Data":"de23e00d2303fdb9cf19868efb73660b44c9a5247d17fd520cde5cba0b319c2c"}
Sep 06 05:15:51.232924 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.232865    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw" event={"ID":"8d8a61c9-f036-4719-a523-5472c6418b0f","Type":"ContainerDied","Data":"976b5d483a0f1639ddc56e301aaf66de8e687eb87aada97f063c9dbe6e9aa1bb"}
Sep 06 05:15:51.232924 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.232893    2578 scope.go:117] "RemoveContainer" containerID="de23e00d2303fdb9cf19868efb73660b44c9a5247d17fd520cde5cba0b319c2c"
Sep 06 05:15:51.243965 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.243942    2578 scope.go:117] "RemoveContainer" containerID="f372c7ceeb78ecc3f899e316c71f67b5738252ebeb54d3d87183d1b443e2f38b"
Sep 06 05:15:51.247407 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.247385    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"]
Sep 06 05:15:51.251099 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.251071    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-241a3769-openshift-default-5585d55f76-lpqvw"]
Sep 06 05:15:51.253330 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.253314    2578 scope.go:117] "RemoveContainer" containerID="de23e00d2303fdb9cf19868efb73660b44c9a5247d17fd520cde5cba0b319c2c"
Sep 06 05:15:51.253597 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:15:51.253579    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de23e00d2303fdb9cf19868efb73660b44c9a5247d17fd520cde5cba0b319c2c\": container with ID starting with de23e00d2303fdb9cf19868efb73660b44c9a5247d17fd520cde5cba0b319c2c not found: ID does not exist" containerID="de23e00d2303fdb9cf19868efb73660b44c9a5247d17fd520cde5cba0b319c2c"
Sep 06 05:15:51.253660 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.253608    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de23e00d2303fdb9cf19868efb73660b44c9a5247d17fd520cde5cba0b319c2c"} err="failed to get container status \"de23e00d2303fdb9cf19868efb73660b44c9a5247d17fd520cde5cba0b319c2c\": rpc error: code = NotFound desc = could not find container \"de23e00d2303fdb9cf19868efb73660b44c9a5247d17fd520cde5cba0b319c2c\": container with ID starting with de23e00d2303fdb9cf19868efb73660b44c9a5247d17fd520cde5cba0b319c2c not found: ID does not exist"
Sep 06 05:15:51.253660 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.253633    2578 scope.go:117] "RemoveContainer" containerID="f372c7ceeb78ecc3f899e316c71f67b5738252ebeb54d3d87183d1b443e2f38b"
Sep 06 05:15:51.253896 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:15:51.253878    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f372c7ceeb78ecc3f899e316c71f67b5738252ebeb54d3d87183d1b443e2f38b\": container with ID starting with f372c7ceeb78ecc3f899e316c71f67b5738252ebeb54d3d87183d1b443e2f38b not found: ID does not exist" containerID="f372c7ceeb78ecc3f899e316c71f67b5738252ebeb54d3d87183d1b443e2f38b"
Sep 06 05:15:51.253943 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:51.253899    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f372c7ceeb78ecc3f899e316c71f67b5738252ebeb54d3d87183d1b443e2f38b"} err="failed to get container status \"f372c7ceeb78ecc3f899e316c71f67b5738252ebeb54d3d87183d1b443e2f38b\": rpc error: code = NotFound desc = could not find container \"f372c7ceeb78ecc3f899e316c71f67b5738252ebeb54d3d87183d1b443e2f38b\": container with ID starting with f372c7ceeb78ecc3f899e316c71f67b5738252ebeb54d3d87183d1b443e2f38b not found: ID does not exist"
Sep 06 05:15:52.012233 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:52.012193    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:52.072978 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:52.072928    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:52.147031 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:52.146986    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:52.249077 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:52.249038    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:52.329386 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:52.329301    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"]
Sep 06 05:15:52.348210 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:52.348185    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:52.432203 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:52.432174    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:52.490175 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:52.490145    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:52.662223 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:52.662177    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:52.772459 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:52.772432    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:53.154759 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:53.154726    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d8a61c9-f036-4719-a523-5472c6418b0f" path="/var/lib/kubelet/pods/8d8a61c9-f036-4719-a523-5472c6418b0f/volumes"
Sep 06 05:15:53.725696 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:53.725649    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"]
Sep 06 05:15:53.744513 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:53.744482    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:54.062898 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:54.062816    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:57.792859 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:57.792825    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:58.003207 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:58.003172    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:58.103732 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:58.103651    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:58.886783 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:58.886754    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:59.143699 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.143625    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"]
Sep 06 05:15:59.144375 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.144349    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d8a61c9-f036-4719-a523-5472c6418b0f" containerName="istio-proxy"
Sep 06 05:15:59.144520 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.144494    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d8a61c9-f036-4719-a523-5472c6418b0f" containerName="istio-proxy"
Sep 06 05:15:59.144598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.144543    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d8a61c9-f036-4719-a523-5472c6418b0f" containerName="istio-proxy"
Sep 06 05:15:59.144598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.144554    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d8a61c9-f036-4719-a523-5472c6418b0f" containerName="istio-proxy"
Sep 06 05:15:59.144705 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.144675    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d8a61c9-f036-4719-a523-5472c6418b0f" containerName="istio-proxy"
Sep 06 05:15:59.144705 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.144689    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d8a61c9-f036-4719-a523-5472c6418b0f" containerName="istio-proxy"
Sep 06 05:15:59.144705 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.144702    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f2e616d-3419-43f5-853b-5f844573fcdb" containerName="cleanup"
Sep 06 05:15:59.149285 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.149260    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.151544 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.151518    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-e24961-openshift-default-dockercfg-wkt6r\""
Sep 06 05:15:59.158489 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.158462    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"]
Sep 06 05:15:59.164964 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.164934    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:15:59.306351 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.306317    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-workload-socket\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.306523 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.306359    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9dsj\" (UniqueName: \"kubernetes.io/projected/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-kube-api-access-d9dsj\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.306523 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.306386    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-envoy\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.306523 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.306492    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-workload-certs\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.306642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.306546    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-data\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.306642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.306563    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-podinfo\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.306642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.306589    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-token\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.306741 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.306657    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-credential-socket\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.306741 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.306681    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istiod-ca-cert\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.407707 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.407622    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-credential-socket\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.407707 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.407666    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istiod-ca-cert\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.407707 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.407698    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-workload-socket\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.407991 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.407722    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9dsj\" (UniqueName: \"kubernetes.io/projected/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-kube-api-access-d9dsj\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.407991 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.407741    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-envoy\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.407991 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.407779    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-workload-certs\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.407991 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.407821    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-data\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.407991 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.407843    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-podinfo\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.407991 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.407869    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-token\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.408308 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.408151    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-credential-socket\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.408308 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.408269    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-workload-certs\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.408400 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.408272    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-workload-socket\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.408443 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.408405    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-data\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.408543 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.408525    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istiod-ca-cert\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.410213 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.410186    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-envoy\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.410442 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.410426    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-podinfo\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.415402 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.415377    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-token\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.415748 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.415728    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9dsj\" (UniqueName: \"kubernetes.io/projected/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-kube-api-access-d9dsj\") pod \"e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") " pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.462810 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.462772    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:15:59.736475 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.736446    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"]
Sep 06 05:15:59.737909 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:15:59.737882    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee44c28c_f2b1_4e58_a413_6fa792dcf4e9.slice/crio-1454b99bda9a25fdd3c71dd175473ebadbb5d875080238f3cadb7ffc685ee0f7 WatchSource:0}: Error finding container 1454b99bda9a25fdd3c71dd175473ebadbb5d875080238f3cadb7ffc685ee0f7: Status 404 returned error can't find the container with id 1454b99bda9a25fdd3c71dd175473ebadbb5d875080238f3cadb7ffc685ee0f7
Sep 06 05:15:59.740128 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.740092    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:15:59.740216 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.740165    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:15:59.740216 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.740198    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:15:59.813061 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:15:59.812990    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:00.282643 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:00.282607    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx" event={"ID":"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9","Type":"ContainerStarted","Data":"a744440c57adb1f90665578a79fd97e24b101678417cdbaf1fa5baa1d02e9843"}
Sep 06 05:16:00.283050 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:00.282668    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx" event={"ID":"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9","Type":"ContainerStarted","Data":"1454b99bda9a25fdd3c71dd175473ebadbb5d875080238f3cadb7ffc685ee0f7"}
Sep 06 05:16:00.300739 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:00.300684    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx" podStartSLOduration=1.3006678329999999 podStartE2EDuration="1.300667833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:15:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:16:00.298943177 +0000 UTC m=+1227.668398424" watchObservedRunningTime="2026-09-06 05:16:00.300667833 +0000 UTC m=+1227.670123079"
Sep 06 05:16:00.463476 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:00.463438    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:16:00.468572 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:00.468547    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:16:01.288335 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:01.288292    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:16:01.289364 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:01.289342    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:16:02.158396 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:02.158361    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:02.249051 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:02.249008    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:03.378198 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:03.378158    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:03.431561 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:03.431508    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:03.543228 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:03.543185    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:03.807603 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:03.807522    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:03.906053 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:03.906000    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:04.075461 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.075376    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"]
Sep 06 05:16:04.075635 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.075613    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr" podUID="e5493df2-7b81-4bab-a62b-9b3c9bfd0d36" containerName="maas-api" containerID="cri-o://621a572ec0851e48a669583d7c031bd01e81eaa81957960e9ea56585a2cb90ca" gracePeriod=30
Sep 06 05:16:04.104315 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.104282    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:04.174639 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.174589    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:04.225727 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.225689    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:04.318631 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.318530    2578 generic.go:356] "Generic (PLEG): container finished" podID="e5493df2-7b81-4bab-a62b-9b3c9bfd0d36" containerID="621a572ec0851e48a669583d7c031bd01e81eaa81957960e9ea56585a2cb90ca" exitCode=0
Sep 06 05:16:04.318631 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.318600    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr" event={"ID":"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36","Type":"ContainerDied","Data":"621a572ec0851e48a669583d7c031bd01e81eaa81957960e9ea56585a2cb90ca"}
Sep 06 05:16:04.327256 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.327197    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:04.350593 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.350571    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:16:04.432256 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.432222    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:04.459298 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.459267    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-maas-api-tls\") pod \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") "
Sep 06 05:16:04.459459 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.459349    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-openshift-service-ca\") pod \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") "
Sep 06 05:16:04.459459 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.459383    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljq9g\" (UniqueName: \"kubernetes.io/projected/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-kube-api-access-ljq9g\") pod \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") "
Sep 06 05:16:04.459459 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.459454    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-metrics-certs\") pod \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\" (UID: \"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36\") "
Sep 06 05:16:04.459727 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.459700    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e5493df2-7b81-4bab-a62b-9b3c9bfd0d36" (UID: "e5493df2-7b81-4bab-a62b-9b3c9bfd0d36"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:16:04.461574 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.461544    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e5493df2-7b81-4bab-a62b-9b3c9bfd0d36" (UID: "e5493df2-7b81-4bab-a62b-9b3c9bfd0d36"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:16:04.461679 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.461595    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e5493df2-7b81-4bab-a62b-9b3c9bfd0d36" (UID: "e5493df2-7b81-4bab-a62b-9b3c9bfd0d36"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:16:04.461679 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.461600    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-kube-api-access-ljq9g" (OuterVolumeSpecName: "kube-api-access-ljq9g") pod "e5493df2-7b81-4bab-a62b-9b3c9bfd0d36" (UID: "e5493df2-7b81-4bab-a62b-9b3c9bfd0d36"). InnerVolumeSpecName "kube-api-access-ljq9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:16:04.560444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.560401    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-metrics-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:04.560444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.560436    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-maas-api-tls\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:04.560444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.560450    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-openshift-service-ca\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:04.560686 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.560463    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljq9g\" (UniqueName: \"kubernetes.io/projected/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36-kube-api-access-ljq9g\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:04.770712 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:04.770670    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:05.325548 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:05.325520    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"
Sep 06 05:16:05.325711 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:05.325515    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr" event={"ID":"e5493df2-7b81-4bab-a62b-9b3c9bfd0d36","Type":"ContainerDied","Data":"2ccd6f646a36f3b14e22dd605f6d3c68017915028976073627dfb503e0d3ed54"}
Sep 06 05:16:05.325711 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:05.325643    2578 scope.go:117] "RemoveContainer" containerID="621a572ec0851e48a669583d7c031bd01e81eaa81957960e9ea56585a2cb90ca"
Sep 06 05:16:05.341427 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:05.341402    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"]
Sep 06 05:16:05.345126 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:05.345104    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr"]
Sep 06 05:16:07.153893 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.153860    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5493df2-7b81-4bab-a62b-9b3c9bfd0d36" path="/var/lib/kubelet/pods/e5493df2-7b81-4bab-a62b-9b3c9bfd0d36/volumes"
Sep 06 05:16:07.313549 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.313506    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3a1cb-95848dfdd-ttwvr" podUID="e5493df2-7b81-4bab-a62b-9b3c9bfd0d36" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.47:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 06 05:16:07.349299 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.349266    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb" podUID="4673dad9-60b4-4370-a0b5-d24616630922" containerName="main" containerID="cri-o://cb734ac621e5f09ea544619aa073af26a5d2b8e2fa627fe93405e92e5d17bd38" gracePeriod=45
Sep 06 05:16:07.619896 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.619870    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:16:07.794787 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.794692    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-dshm\") pod \"4673dad9-60b4-4370-a0b5-d24616630922\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") "
Sep 06 05:16:07.794787 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.794758    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c87rd\" (UniqueName: \"kubernetes.io/projected/4673dad9-60b4-4370-a0b5-d24616630922-kube-api-access-c87rd\") pod \"4673dad9-60b4-4370-a0b5-d24616630922\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") "
Sep 06 05:16:07.795055 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.794811    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4673dad9-60b4-4370-a0b5-d24616630922-tls-certs\") pod \"4673dad9-60b4-4370-a0b5-d24616630922\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") "
Sep 06 05:16:07.795055 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.794935    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-tmp-dir\") pod \"4673dad9-60b4-4370-a0b5-d24616630922\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") "
Sep 06 05:16:07.795055 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.795004    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-model-cache\") pod \"4673dad9-60b4-4370-a0b5-d24616630922\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") "
Sep 06 05:16:07.795212 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.795056    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-home\") pod \"4673dad9-60b4-4370-a0b5-d24616630922\" (UID: \"4673dad9-60b4-4370-a0b5-d24616630922\") "
Sep 06 05:16:07.795311 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.795268    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4673dad9-60b4-4370-a0b5-d24616630922" (UID: "4673dad9-60b4-4370-a0b5-d24616630922"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:07.795311 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.795293    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-model-cache" (OuterVolumeSpecName: "model-cache") pod "4673dad9-60b4-4370-a0b5-d24616630922" (UID: "4673dad9-60b4-4370-a0b5-d24616630922"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:07.795516 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.795463    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-tmp-dir\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:07.795516 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.795485    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-model-cache\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:07.795516 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.795481    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-home" (OuterVolumeSpecName: "home") pod "4673dad9-60b4-4370-a0b5-d24616630922" (UID: "4673dad9-60b4-4370-a0b5-d24616630922"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:07.797191 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.797165    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4673dad9-60b4-4370-a0b5-d24616630922-kube-api-access-c87rd" (OuterVolumeSpecName: "kube-api-access-c87rd") pod "4673dad9-60b4-4370-a0b5-d24616630922" (UID: "4673dad9-60b4-4370-a0b5-d24616630922"). InnerVolumeSpecName "kube-api-access-c87rd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:16:07.797365 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.797347    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-dshm" (OuterVolumeSpecName: "dshm") pod "4673dad9-60b4-4370-a0b5-d24616630922" (UID: "4673dad9-60b4-4370-a0b5-d24616630922"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:07.797419 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.797370    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4673dad9-60b4-4370-a0b5-d24616630922-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4673dad9-60b4-4370-a0b5-d24616630922" (UID: "4673dad9-60b4-4370-a0b5-d24616630922"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:16:07.896121 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.896079    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-home\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:07.896121 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.896111    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4673dad9-60b4-4370-a0b5-d24616630922-dshm\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:07.896121 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.896120    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c87rd\" (UniqueName: \"kubernetes.io/projected/4673dad9-60b4-4370-a0b5-d24616630922-kube-api-access-c87rd\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:07.896121 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:07.896130    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4673dad9-60b4-4370-a0b5-d24616630922-tls-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:08.343185 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:08.343148    2578 generic.go:356] "Generic (PLEG): container finished" podID="4673dad9-60b4-4370-a0b5-d24616630922" containerID="cb734ac621e5f09ea544619aa073af26a5d2b8e2fa627fe93405e92e5d17bd38" exitCode=0
Sep 06 05:16:08.343587 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:08.343196    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"
Sep 06 05:16:08.343587 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:08.343216    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb" event={"ID":"4673dad9-60b4-4370-a0b5-d24616630922","Type":"ContainerDied","Data":"cb734ac621e5f09ea544619aa073af26a5d2b8e2fa627fe93405e92e5d17bd38"}
Sep 06 05:16:08.343587 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:08.343256    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb" event={"ID":"4673dad9-60b4-4370-a0b5-d24616630922","Type":"ContainerDied","Data":"33bda18bfea859f1c7c2b0b81a636107f76d804bf8a1cc26aee202b0a9666b64"}
Sep 06 05:16:08.343587 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:08.343278    2578 scope.go:117] "RemoveContainer" containerID="cb734ac621e5f09ea544619aa073af26a5d2b8e2fa627fe93405e92e5d17bd38"
Sep 06 05:16:08.352555 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:08.352371    2578 scope.go:117] "RemoveContainer" containerID="cb734ac621e5f09ea544619aa073af26a5d2b8e2fa627fe93405e92e5d17bd38"
Sep 06 05:16:08.352705 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:16:08.352684    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb734ac621e5f09ea544619aa073af26a5d2b8e2fa627fe93405e92e5d17bd38\": container with ID starting with cb734ac621e5f09ea544619aa073af26a5d2b8e2fa627fe93405e92e5d17bd38 not found: ID does not exist" containerID="cb734ac621e5f09ea544619aa073af26a5d2b8e2fa627fe93405e92e5d17bd38"
Sep 06 05:16:08.352753 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:08.352716    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb734ac621e5f09ea544619aa073af26a5d2b8e2fa627fe93405e92e5d17bd38"} err="failed to get container status \"cb734ac621e5f09ea544619aa073af26a5d2b8e2fa627fe93405e92e5d17bd38\": rpc error: code = NotFound desc = could not find container \"cb734ac621e5f09ea544619aa073af26a5d2b8e2fa627fe93405e92e5d17bd38\": container with ID starting with cb734ac621e5f09ea544619aa073af26a5d2b8e2fa627fe93405e92e5d17bd38 not found: ID does not exist"
Sep 06 05:16:08.363094 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:08.363065    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"]
Sep 06 05:16:08.366463 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:08.366440    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-c53189/noten-model-c53189-kserve-f76657795-tb8qb"]
Sep 06 05:16:08.745905 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:08.745843    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt" podUID="866c379d-59e1-4ca1-85c7-7dad7a3b8710" containerName="main" containerID="cri-o://a2179bb47de75866c56c259d6a640018b7dee4b2dff1b10c3459c00e567cf2e5" gracePeriod=45
Sep 06 05:16:09.004545 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.004483    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:16:09.107182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.107150    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5pqw\" (UniqueName: \"kubernetes.io/projected/866c379d-59e1-4ca1-85c7-7dad7a3b8710-kube-api-access-j5pqw\") pod \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") "
Sep 06 05:16:09.107358 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.107217    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-dshm\") pod \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") "
Sep 06 05:16:09.107358 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.107263    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/866c379d-59e1-4ca1-85c7-7dad7a3b8710-tls-certs\") pod \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") "
Sep 06 05:16:09.107358 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.107302    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-tmp-dir\") pod \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") "
Sep 06 05:16:09.107358 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.107321    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-home\") pod \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") "
Sep 06 05:16:09.107585 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.107369    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-model-cache\") pod \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\" (UID: \"866c379d-59e1-4ca1-85c7-7dad7a3b8710\") "
Sep 06 05:16:09.107585 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.107565    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "866c379d-59e1-4ca1-85c7-7dad7a3b8710" (UID: "866c379d-59e1-4ca1-85c7-7dad7a3b8710"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:09.107700 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.107601    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-home" (OuterVolumeSpecName: "home") pod "866c379d-59e1-4ca1-85c7-7dad7a3b8710" (UID: "866c379d-59e1-4ca1-85c7-7dad7a3b8710"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:09.107700 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.107678    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-model-cache" (OuterVolumeSpecName: "model-cache") pod "866c379d-59e1-4ca1-85c7-7dad7a3b8710" (UID: "866c379d-59e1-4ca1-85c7-7dad7a3b8710"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:09.109494 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.109471    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-dshm" (OuterVolumeSpecName: "dshm") pod "866c379d-59e1-4ca1-85c7-7dad7a3b8710" (UID: "866c379d-59e1-4ca1-85c7-7dad7a3b8710"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:09.109587 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.109566    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/866c379d-59e1-4ca1-85c7-7dad7a3b8710-kube-api-access-j5pqw" (OuterVolumeSpecName: "kube-api-access-j5pqw") pod "866c379d-59e1-4ca1-85c7-7dad7a3b8710" (UID: "866c379d-59e1-4ca1-85c7-7dad7a3b8710"). InnerVolumeSpecName "kube-api-access-j5pqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:16:09.109626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.109574    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/866c379d-59e1-4ca1-85c7-7dad7a3b8710-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "866c379d-59e1-4ca1-85c7-7dad7a3b8710" (UID: "866c379d-59e1-4ca1-85c7-7dad7a3b8710"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:16:09.153105 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.153079    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4673dad9-60b4-4370-a0b5-d24616630922" path="/var/lib/kubelet/pods/4673dad9-60b4-4370-a0b5-d24616630922/volumes"
Sep 06 05:16:09.209649 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.209604    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-tmp-dir\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:09.209649 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.209643    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-home\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:09.209649 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.209658    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-model-cache\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:09.209649 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.209672    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5pqw\" (UniqueName: \"kubernetes.io/projected/866c379d-59e1-4ca1-85c7-7dad7a3b8710-kube-api-access-j5pqw\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:09.210096 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.209688    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/866c379d-59e1-4ca1-85c7-7dad7a3b8710-dshm\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:09.210096 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.209702    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/866c379d-59e1-4ca1-85c7-7dad7a3b8710-tls-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:09.350712 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.350622    2578 generic.go:356] "Generic (PLEG): container finished" podID="866c379d-59e1-4ca1-85c7-7dad7a3b8710" containerID="a2179bb47de75866c56c259d6a640018b7dee4b2dff1b10c3459c00e567cf2e5" exitCode=0
Sep 06 05:16:09.350712 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.350668    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"
Sep 06 05:16:09.351227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.350706    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt" event={"ID":"866c379d-59e1-4ca1-85c7-7dad7a3b8710","Type":"ContainerDied","Data":"a2179bb47de75866c56c259d6a640018b7dee4b2dff1b10c3459c00e567cf2e5"}
Sep 06 05:16:09.351227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.350753    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt" event={"ID":"866c379d-59e1-4ca1-85c7-7dad7a3b8710","Type":"ContainerDied","Data":"2985a65dce12131c0787d141742f509509a753fb40f667c6311dffb826174016"}
Sep 06 05:16:09.351227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.350770    2578 scope.go:117] "RemoveContainer" containerID="a2179bb47de75866c56c259d6a640018b7dee4b2dff1b10c3459c00e567cf2e5"
Sep 06 05:16:09.359177 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.358981    2578 scope.go:117] "RemoveContainer" containerID="a2179bb47de75866c56c259d6a640018b7dee4b2dff1b10c3459c00e567cf2e5"
Sep 06 05:16:09.359263 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:16:09.359245    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2179bb47de75866c56c259d6a640018b7dee4b2dff1b10c3459c00e567cf2e5\": container with ID starting with a2179bb47de75866c56c259d6a640018b7dee4b2dff1b10c3459c00e567cf2e5 not found: ID does not exist" containerID="a2179bb47de75866c56c259d6a640018b7dee4b2dff1b10c3459c00e567cf2e5"
Sep 06 05:16:09.359307 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.359270    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2179bb47de75866c56c259d6a640018b7dee4b2dff1b10c3459c00e567cf2e5"} err="failed to get container status \"a2179bb47de75866c56c259d6a640018b7dee4b2dff1b10c3459c00e567cf2e5\": rpc error: code = NotFound desc = could not find container \"a2179bb47de75866c56c259d6a640018b7dee4b2dff1b10c3459c00e567cf2e5\": container with ID starting with a2179bb47de75866c56c259d6a640018b7dee4b2dff1b10c3459c00e567cf2e5 not found: ID does not exist"
Sep 06 05:16:09.367526 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.367504    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"]
Sep 06 05:16:09.369276 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:09.369258    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-9035bc/test-model-9035bc-kserve-58bcb6b68f-2f8vt"]
Sep 06 05:16:10.491824 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:10.491789    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:10.651648 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:10.651606    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:10.777003 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:10.776904    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"]
Sep 06 05:16:10.777198 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:10.777171    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47" podUID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" containerName="istio-proxy" containerID="cri-o://03714524cf6edc2601b41fef2ea4ac530d87e5b6ff551b67184a910a8995785d" gracePeriod=30
Sep 06 05:16:10.808040 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:10.799408    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:11.154339 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:11.154307    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="866c379d-59e1-4ca1-85c7-7dad7a3b8710" path="/var/lib/kubelet/pods/866c379d-59e1-4ca1-85c7-7dad7a3b8710/volumes"
Sep 06 05:16:11.266809 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:11.266777    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:16.084918 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.084885    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:16:16.175959 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.175923    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-token\") pod \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") "
Sep 06 05:16:16.176175 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.175970    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-workload-socket\") pod \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") "
Sep 06 05:16:16.176175 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.175996    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-credential-socket\") pod \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") "
Sep 06 05:16:16.176175 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176045    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-data\") pod \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") "
Sep 06 05:16:16.176175 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176075    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-envoy\") pod \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") "
Sep 06 05:16:16.176175 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176107    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-podinfo\") pod \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") "
Sep 06 05:16:16.176175 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176163    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f4d9\" (UniqueName: \"kubernetes.io/projected/e2a26e4c-c0f3-4851-b505-febe3bc6806e-kube-api-access-6f4d9\") pod \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") "
Sep 06 05:16:16.176472 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176231    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-workload-certs\") pod \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") "
Sep 06 05:16:16.176472 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176246    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e2a26e4c-c0f3-4851-b505-febe3bc6806e" (UID: "e2a26e4c-c0f3-4851-b505-febe3bc6806e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:16.176472 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176266    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istiod-ca-cert\") pod \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\" (UID: \"e2a26e4c-c0f3-4851-b505-febe3bc6806e\") "
Sep 06 05:16:16.176472 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176262    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-data" (OuterVolumeSpecName: "istio-data") pod "e2a26e4c-c0f3-4851-b505-febe3bc6806e" (UID: "e2a26e4c-c0f3-4851-b505-febe3bc6806e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:16.176472 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176290    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e2a26e4c-c0f3-4851-b505-febe3bc6806e" (UID: "e2a26e4c-c0f3-4851-b505-febe3bc6806e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:16.176737 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176556    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e2a26e4c-c0f3-4851-b505-febe3bc6806e" (UID: "e2a26e4c-c0f3-4851-b505-febe3bc6806e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:16.176737 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176677    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-workload-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:16.176737 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176680    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e2a26e4c-c0f3-4851-b505-febe3bc6806e" (UID: "e2a26e4c-c0f3-4851-b505-febe3bc6806e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:16:16.176737 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176697    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-credential-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:16.176737 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176712    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-data\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:16.176737 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.176726    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-workload-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:16.178293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.178266    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-token" (OuterVolumeSpecName: "istio-token") pod "e2a26e4c-c0f3-4851-b505-febe3bc6806e" (UID: "e2a26e4c-c0f3-4851-b505-febe3bc6806e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:16:16.178402 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.178356    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e2a26e4c-c0f3-4851-b505-febe3bc6806e" (UID: "e2a26e4c-c0f3-4851-b505-febe3bc6806e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:16.178460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.178423    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e2a26e4c-c0f3-4851-b505-febe3bc6806e" (UID: "e2a26e4c-c0f3-4851-b505-febe3bc6806e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:16:16.178460 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.178426    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2a26e4c-c0f3-4851-b505-febe3bc6806e-kube-api-access-6f4d9" (OuterVolumeSpecName: "kube-api-access-6f4d9") pod "e2a26e4c-c0f3-4851-b505-febe3bc6806e" (UID: "e2a26e4c-c0f3-4851-b505-febe3bc6806e"). InnerVolumeSpecName "kube-api-access-6f4d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:16:16.277648 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.277608    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istiod-ca-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:16.277648 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.277641    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-token\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:16.277648 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.277649    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-envoy\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:16.277855 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.277656    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2a26e4c-c0f3-4851-b505-febe3bc6806e-istio-podinfo\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:16.277855 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.277666    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f4d9\" (UniqueName: \"kubernetes.io/projected/e2a26e4c-c0f3-4851-b505-febe3bc6806e-kube-api-access-6f4d9\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:16.392614 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.392575    2578 generic.go:356] "Generic (PLEG): container finished" podID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" containerID="03714524cf6edc2601b41fef2ea4ac530d87e5b6ff551b67184a910a8995785d" exitCode=0
Sep 06 05:16:16.392821 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.392624    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"
Sep 06 05:16:16.392821 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.392653    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47" event={"ID":"e2a26e4c-c0f3-4851-b505-febe3bc6806e","Type":"ContainerDied","Data":"03714524cf6edc2601b41fef2ea4ac530d87e5b6ff551b67184a910a8995785d"}
Sep 06 05:16:16.392821 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.392718    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47" event={"ID":"e2a26e4c-c0f3-4851-b505-febe3bc6806e","Type":"ContainerDied","Data":"fde9079706bc11dbad500612ff48c6d9006d8d1be65d6e0f12bef9a6a2f9153e"}
Sep 06 05:16:16.392821 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.392747    2578 scope.go:117] "RemoveContainer" containerID="03714524cf6edc2601b41fef2ea4ac530d87e5b6ff551b67184a910a8995785d"
Sep 06 05:16:16.402503 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.402488    2578 scope.go:117] "RemoveContainer" containerID="48006ff27cc35c9a0f83dea008982324a19401fdc58da60a58ffdf4aab7aa4db"
Sep 06 05:16:16.410671 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.410654    2578 scope.go:117] "RemoveContainer" containerID="03714524cf6edc2601b41fef2ea4ac530d87e5b6ff551b67184a910a8995785d"
Sep 06 05:16:16.410917 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:16:16.410901    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03714524cf6edc2601b41fef2ea4ac530d87e5b6ff551b67184a910a8995785d\": container with ID starting with 03714524cf6edc2601b41fef2ea4ac530d87e5b6ff551b67184a910a8995785d not found: ID does not exist" containerID="03714524cf6edc2601b41fef2ea4ac530d87e5b6ff551b67184a910a8995785d"
Sep 06 05:16:16.410963 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.410923    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03714524cf6edc2601b41fef2ea4ac530d87e5b6ff551b67184a910a8995785d"} err="failed to get container status \"03714524cf6edc2601b41fef2ea4ac530d87e5b6ff551b67184a910a8995785d\": rpc error: code = NotFound desc = could not find container \"03714524cf6edc2601b41fef2ea4ac530d87e5b6ff551b67184a910a8995785d\": container with ID starting with 03714524cf6edc2601b41fef2ea4ac530d87e5b6ff551b67184a910a8995785d not found: ID does not exist"
Sep 06 05:16:16.410963 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.410940    2578 scope.go:117] "RemoveContainer" containerID="48006ff27cc35c9a0f83dea008982324a19401fdc58da60a58ffdf4aab7aa4db"
Sep 06 05:16:16.411198 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:16:16.411181    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48006ff27cc35c9a0f83dea008982324a19401fdc58da60a58ffdf4aab7aa4db\": container with ID starting with 48006ff27cc35c9a0f83dea008982324a19401fdc58da60a58ffdf4aab7aa4db not found: ID does not exist" containerID="48006ff27cc35c9a0f83dea008982324a19401fdc58da60a58ffdf4aab7aa4db"
Sep 06 05:16:16.411315 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.411205    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48006ff27cc35c9a0f83dea008982324a19401fdc58da60a58ffdf4aab7aa4db"} err="failed to get container status \"48006ff27cc35c9a0f83dea008982324a19401fdc58da60a58ffdf4aab7aa4db\": rpc error: code = NotFound desc = could not find container \"48006ff27cc35c9a0f83dea008982324a19401fdc58da60a58ffdf4aab7aa4db\": container with ID starting with 48006ff27cc35c9a0f83dea008982324a19401fdc58da60a58ffdf4aab7aa4db not found: ID does not exist"
Sep 06 05:16:16.414220 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.414198    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"]
Sep 06 05:16:16.417489 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:16.417471    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-d3a1cb-openshift-default-5bc496bd9d-q9f47"]
Sep 06 05:16:17.154846 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:17.154806    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" path="/var/lib/kubelet/pods/e2a26e4c-c0f3-4851-b505-febe3bc6806e/volumes"
Sep 06 05:16:17.330233 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:17.330194    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 05:16:17.330451 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:17.330269    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx" podUID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 05:16:19.413537 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:19.413449    2578 generic.go:356] "Generic (PLEG): container finished" podID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerID="a744440c57adb1f90665578a79fd97e24b101678417cdbaf1fa5baa1d02e9843" exitCode=0
Sep 06 05:16:19.413920 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:19.413529    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx" event={"ID":"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9","Type":"ContainerDied","Data":"a744440c57adb1f90665578a79fd97e24b101678417cdbaf1fa5baa1d02e9843"}
Sep 06 05:16:19.413974 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:19.413929    2578 scope.go:117] "RemoveContainer" containerID="a744440c57adb1f90665578a79fd97e24b101678417cdbaf1fa5baa1d02e9843"
Sep 06 05:16:19.414965 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:19.414939    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:16:19.415043 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:19.415005    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:16:19.415088 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:19.415058    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:16:20.422269 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:20.422236    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx" event={"ID":"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9","Type":"ContainerStarted","Data":"752455ebc506e2e5791ff0ebb0ed5da8086f298f6f3b5a74edb254fd8391314f"}
Sep 06 05:16:20.463118 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:20.463083    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:16:20.463323 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:20.463305    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 06 05:16:20.463371 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:20.463346    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx" podUID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 06 05:16:21.463610 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:21.463574    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 06 05:16:21.464047 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:21.463633    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx" podUID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 06 05:16:21.726682 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:21.726590    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:21.803956 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:21.803892    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:21.893668 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:21.893569    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:21.957279 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:21.957239    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:22.258473 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:22.258442    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:22.463803 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:22.463762    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 06 05:16:22.464216 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:22.463822    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx" podUID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 06 05:16:23.463287 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:23.463250    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 06 05:16:23.463466 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:23.463312    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx" podUID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 06 05:16:24.192831 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:24.192771    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:24.463959 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:24.463874    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 06 05:16:24.463959 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:24.463938    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx" podUID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 06 05:16:25.705457 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:25.705367    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:16:25.706109 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:25.706077    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:16:25.707729 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:25.707709    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:16:27.073694 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:27.073660    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:27.393076 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:27.393038    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:27.675781 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:27.675702    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:27.775170 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:27.775134    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:28.458549 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:28.458510    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:28.526760 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:28.526725    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:28.691673 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:28.691639    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"]
Sep 06 05:16:28.692211 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:28.692176    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm" podUID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" containerName="istio-proxy" containerID="cri-o://f58c3b85566824b52ed68806973f645798c7319a343378559e8e6c886f796068" gracePeriod=30
Sep 06 05:16:28.713938 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:28.713841    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:29.258290 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.258254    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:29.527008 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.526926    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"]
Sep 06 05:16:29.527479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.527344    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4673dad9-60b4-4370-a0b5-d24616630922" containerName="main"
Sep 06 05:16:29.527479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.527357    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4673dad9-60b4-4370-a0b5-d24616630922" containerName="main"
Sep 06 05:16:29.527479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.527375    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" containerName="istio-proxy"
Sep 06 05:16:29.527479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.527381    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" containerName="istio-proxy"
Sep 06 05:16:29.527479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.527391    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" containerName="istio-proxy"
Sep 06 05:16:29.527479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.527396    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" containerName="istio-proxy"
Sep 06 05:16:29.527479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.527407    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="866c379d-59e1-4ca1-85c7-7dad7a3b8710" containerName="main"
Sep 06 05:16:29.527479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.527412    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="866c379d-59e1-4ca1-85c7-7dad7a3b8710" containerName="main"
Sep 06 05:16:29.527479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.527420    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5493df2-7b81-4bab-a62b-9b3c9bfd0d36" containerName="maas-api"
Sep 06 05:16:29.527479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.527425    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5493df2-7b81-4bab-a62b-9b3c9bfd0d36" containerName="maas-api"
Sep 06 05:16:29.527479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.527479    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" containerName="istio-proxy"
Sep 06 05:16:29.527479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.527488    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4673dad9-60b4-4370-a0b5-d24616630922" containerName="main"
Sep 06 05:16:29.528144 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.527495    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="866c379d-59e1-4ca1-85c7-7dad7a3b8710" containerName="main"
Sep 06 05:16:29.528144 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.527502    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5493df2-7b81-4bab-a62b-9b3c9bfd0d36" containerName="maas-api"
Sep 06 05:16:29.530788 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.530769    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.533046 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.533005    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-494740-openshift-default-dockercfg-9ncf8\""
Sep 06 05:16:29.541184 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.541155    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"]
Sep 06 05:16:29.549307 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.549281    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:29.610864 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.610835    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-envoy\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.611062 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.610870    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24s9l\" (UniqueName: \"kubernetes.io/projected/86d5a085-cde4-4d06-ab3d-65f550224a3e-kube-api-access-24s9l\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.611062 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.610899    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-workload-certs\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.611062 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.610953    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-podinfo\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.611062 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.610996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-data\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.611236 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.611095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86d5a085-cde4-4d06-ab3d-65f550224a3e-istiod-ca-cert\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.611236 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.611125    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-token\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.611236 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.611155    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-workload-socket\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.611236 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.611206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-credential-socket\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.712562 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.712527    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-envoy\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.712562 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.712568    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24s9l\" (UniqueName: \"kubernetes.io/projected/86d5a085-cde4-4d06-ab3d-65f550224a3e-kube-api-access-24s9l\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.712817 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.712604    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-workload-certs\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.712817 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.712625    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-podinfo\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.712817 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.712648    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-data\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.712817 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.712715    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86d5a085-cde4-4d06-ab3d-65f550224a3e-istiod-ca-cert\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.712817 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.712751    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-token\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.712817 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.712793    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-workload-socket\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.713208 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.712824    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-credential-socket\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.713208 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.713101    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-workload-certs\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.713343 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.713236    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-data\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.713343 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.713282    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-credential-socket\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.713446 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.713372    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-workload-socket\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.713547 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.713523    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86d5a085-cde4-4d06-ab3d-65f550224a3e-istiod-ca-cert\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.714895 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.714873    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-envoy\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.715136 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.715116    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-podinfo\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.719781 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.719758    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-token\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.719781 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.719770    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24s9l\" (UniqueName: \"kubernetes.io/projected/86d5a085-cde4-4d06-ab3d-65f550224a3e-kube-api-access-24s9l\") pod \"e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") " pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:29.781070 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.780967    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:29.844574 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:29.844544    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:30.144532 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:30.144500    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"]
Sep 06 05:16:30.145805 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:16:30.145777    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d5a085_cde4_4d06_ab3d_65f550224a3e.slice/crio-195de068273f779391213050abca544aaf0b5acd8bcfb5202ad6a54553b6bd4c WatchSource:0}: Error finding container 195de068273f779391213050abca544aaf0b5acd8bcfb5202ad6a54553b6bd4c: Status 404 returned error can't find the container with id 195de068273f779391213050abca544aaf0b5acd8bcfb5202ad6a54553b6bd4c
Sep 06 05:16:30.148173 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:30.148145    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:16:30.148256 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:30.148207    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:16:30.148256 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:30.148239    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:16:30.482230 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:30.482142    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq" event={"ID":"86d5a085-cde4-4d06-ab3d-65f550224a3e","Type":"ContainerStarted","Data":"f51a1464a91f98bc0f9db712a36d9182343e51340b1527584fc339ccd4146f33"}
Sep 06 05:16:30.482230 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:30.482194    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq" event={"ID":"86d5a085-cde4-4d06-ab3d-65f550224a3e","Type":"ContainerStarted","Data":"195de068273f779391213050abca544aaf0b5acd8bcfb5202ad6a54553b6bd4c"}
Sep 06 05:16:30.500651 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:30.500598    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq" podStartSLOduration=1.500577743 podStartE2EDuration="1.500577743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:16:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:16:30.498769834 +0000 UTC m=+1257.868225081" watchObservedRunningTime="2026-09-06 05:16:30.500577743 +0000 UTC m=+1257.870032989"
Sep 06 05:16:30.845191 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:30.845091    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:30.874922 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:30.874889    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:30.974251 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:30.974217    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:31.850906 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:31.850875    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:31.972523 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:31.972485    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"]
Sep 06 05:16:31.973323 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:31.973302    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2a26e4c-c0f3-4851-b505-febe3bc6806e" containerName="istio-proxy"
Sep 06 05:16:31.976993 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:31.976964    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:31.979219 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:31.979194    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-401f17-serving-cert\""
Sep 06 05:16:31.985223 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:31.985194    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"]
Sep 06 05:16:32.002522 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.002489    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:32.038070 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.038004    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43725b3f-a6f4-4f8d-b614-679e019d3cb8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:32.038241 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.038079    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:32.038241 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.038174    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:32.038241 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.038223    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wrwd\" (UniqueName: \"kubernetes.io/projected/43725b3f-a6f4-4f8d-b614-679e019d3cb8-kube-api-access-8wrwd\") pod \"maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:32.053719 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.053685    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:32.138874 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.138835    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:32.139101 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.138890    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wrwd\" (UniqueName: \"kubernetes.io/projected/43725b3f-a6f4-4f8d-b614-679e019d3cb8-kube-api-access-8wrwd\") pod \"maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:32.139101 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.138927    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43725b3f-a6f4-4f8d-b614-679e019d3cb8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:32.139101 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.138949    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:32.139280 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:16:32.139102    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-serving-cert: secret "maas-api-e2e-shared-b-w2-401f17-serving-cert" not found
Sep 06 05:16:32.139280 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:16:32.139175    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-maas-api-tls podName:43725b3f-a6f4-4f8d-b614-679e019d3cb8 nodeName:}" failed. No retries permitted until 2026-09-06 05:16:32.639152809 +0000 UTC m=+1260.008608051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-maas-api-tls") pod "maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58" (UID: "43725b3f-a6f4-4f8d-b614-679e019d3cb8") : secret "maas-api-e2e-shared-b-w2-401f17-serving-cert" not found
Sep 06 05:16:32.139598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.139578    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43725b3f-a6f4-4f8d-b614-679e019d3cb8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:32.141420 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.141398    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:32.147782 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.147748    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wrwd\" (UniqueName: \"kubernetes.io/projected/43725b3f-a6f4-4f8d-b614-679e019d3cb8-kube-api-access-8wrwd\") pod \"maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:32.343560 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.343525    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:32.442880 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.442761    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:32.495038 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.494987    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:32.496064 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.496040    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:32.592439 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.592405    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:32.644078 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.644047    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:32.646403 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.646378    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:32.676443 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.676412    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:32.722741 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.722673    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"]
Sep 06 05:16:32.725996 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.725980    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"
Sep 06 05:16:32.728352 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.728334    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-401f17\""
Sep 06 05:16:32.728431 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.728368    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-401f17-dockercfg-xg5sp\""
Sep 06 05:16:32.739603 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.739577    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"]
Sep 06 05:16:32.845364 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.845330    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-672wm\" (UniqueName: \"kubernetes.io/projected/b3b9b742-ef50-4aac-90a4-01ebbc38590e-kube-api-access-672wm\") pod \"payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x\" (UID: \"b3b9b742-ef50-4aac-90a4-01ebbc38590e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"
Sep 06 05:16:32.845550 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.845443    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3b9b742-ef50-4aac-90a4-01ebbc38590e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x\" (UID: \"b3b9b742-ef50-4aac-90a4-01ebbc38590e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"
Sep 06 05:16:32.899225 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.899187    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:32.946965 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.946934    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-672wm\" (UniqueName: \"kubernetes.io/projected/b3b9b742-ef50-4aac-90a4-01ebbc38590e-kube-api-access-672wm\") pod \"payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x\" (UID: \"b3b9b742-ef50-4aac-90a4-01ebbc38590e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"
Sep 06 05:16:32.947315 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.947291    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3b9b742-ef50-4aac-90a4-01ebbc38590e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x\" (UID: \"b3b9b742-ef50-4aac-90a4-01ebbc38590e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"
Sep 06 05:16:32.949374 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.948151    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3b9b742-ef50-4aac-90a4-01ebbc38590e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x\" (UID: \"b3b9b742-ef50-4aac-90a4-01ebbc38590e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"
Sep 06 05:16:32.956887 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:32.956858    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-672wm\" (UniqueName: \"kubernetes.io/projected/b3b9b742-ef50-4aac-90a4-01ebbc38590e-kube-api-access-672wm\") pod \"payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x\" (UID: \"b3b9b742-ef50-4aac-90a4-01ebbc38590e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"
Sep 06 05:16:33.038611 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:33.038515    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-401f17-dockercfg-xg5sp\""
Sep 06 05:16:33.046530 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:33.046505    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"
Sep 06 05:16:33.171235 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:33.171207    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"]
Sep 06 05:16:33.172866 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:16:33.172838    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43725b3f_a6f4_4f8d_b614_679e019d3cb8.slice/crio-10e4a905d28e912b3e3127811cc61c299266a70e165dd0303631aad075b9c6c6 WatchSource:0}: Error finding container 10e4a905d28e912b3e3127811cc61c299266a70e165dd0303631aad075b9c6c6: Status 404 returned error can't find the container with id 10e4a905d28e912b3e3127811cc61c299266a70e165dd0303631aad075b9c6c6
Sep 06 05:16:33.319533 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:33.319506    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"]
Sep 06 05:16:33.323420 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:16:33.323398    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3b9b742_ef50_4aac_90a4_01ebbc38590e.slice/crio-e7ee0a10fedabc9eb70730cc3af45d9c63369e8c328afc062a95076488b5f315 WatchSource:0}: Error finding container e7ee0a10fedabc9eb70730cc3af45d9c63369e8c328afc062a95076488b5f315: Status 404 returned error can't find the container with id e7ee0a10fedabc9eb70730cc3af45d9c63369e8c328afc062a95076488b5f315
Sep 06 05:16:33.494876 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:33.494843    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:33.501968 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:33.501933    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x" event={"ID":"b3b9b742-ef50-4aac-90a4-01ebbc38590e","Type":"ContainerStarted","Data":"d0d7340f6b5a4b741233aa46dd391820f37a35b9a6517d26e76326e374dd96ad"}
Sep 06 05:16:33.502167 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:33.501987    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x" event={"ID":"b3b9b742-ef50-4aac-90a4-01ebbc38590e","Type":"ContainerStarted","Data":"e7ee0a10fedabc9eb70730cc3af45d9c63369e8c328afc062a95076488b5f315"}
Sep 06 05:16:33.502167 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:33.502093    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"
Sep 06 05:16:33.503710 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:33.503667    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58" event={"ID":"43725b3f-a6f4-4f8d-b614-679e019d3cb8","Type":"ContainerStarted","Data":"10e4a905d28e912b3e3127811cc61c299266a70e165dd0303631aad075b9c6c6"}
Sep 06 05:16:33.519204 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:33.519156    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x" podStartSLOduration=1.519141672 podStartE2EDuration="1.519141672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:16:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:16:33.517082469 +0000 UTC m=+1260.886537716" watchObservedRunningTime="2026-09-06 05:16:33.519141672 +0000 UTC m=+1260.888596944"
Sep 06 05:16:33.596066 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:33.596033    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:33.892652 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:33.892576    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:33.938089 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:33.938061    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:16:34.057263 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057224    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-workload-certs\") pod \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") "
Sep 06 05:16:34.057263 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057269    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-data\") pod \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") "
Sep 06 05:16:34.057503 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057298    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-envoy\") pod \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") "
Sep 06 05:16:34.057503 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057323    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgj9x\" (UniqueName: \"kubernetes.io/projected/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-kube-api-access-cgj9x\") pod \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") "
Sep 06 05:16:34.057503 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057361    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-token\") pod \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") "
Sep 06 05:16:34.057503 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057383    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-credential-socket\") pod \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") "
Sep 06 05:16:34.057503 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057420    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-workload-socket\") pod \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") "
Sep 06 05:16:34.057503 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057488    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istiod-ca-cert\") pod \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") "
Sep 06 05:16:34.057822 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057559    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-podinfo\") pod \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\" (UID: \"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312\") "
Sep 06 05:16:34.057822 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057568    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-data" (OuterVolumeSpecName: "istio-data") pod "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" (UID: "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:34.057822 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057691    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" (UID: "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:34.057822 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057713    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" (UID: "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:34.057822 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057796    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" (UID: "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:34.058125 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057885    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" (UID: "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:16:34.058125 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.057993    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istiod-ca-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:34.058125 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.058045    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-workload-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:34.058125 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.058061    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-data\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:34.058125 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.058075    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-credential-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:34.058125 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.058088    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-workload-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:34.059645 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.059585    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" (UID: "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:16:34.059645 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.059601    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-kube-api-access-cgj9x" (OuterVolumeSpecName: "kube-api-access-cgj9x") pod "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" (UID: "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312"). InnerVolumeSpecName "kube-api-access-cgj9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:16:34.059645 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.059609    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" (UID: "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:16:34.059810 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.059661    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" (UID: "7e6a0a4d-5526-4e50-8fee-49ef9d2d1312"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:16:34.152556 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.152429    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:34.158937 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.158909    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-podinfo\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:34.159072 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.158942    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-envoy\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:34.159072 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.158955    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgj9x\" (UniqueName: \"kubernetes.io/projected/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-kube-api-access-cgj9x\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:34.159072 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.158967    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312-istio-token\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:16:34.253453 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.253413    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:34.393861 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.393595    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"]
Sep 06 05:16:34.394761 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.394249    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" containerName="istio-proxy"
Sep 06 05:16:34.394761 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.394272    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" containerName="istio-proxy"
Sep 06 05:16:34.394761 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.394411    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" containerName="istio-proxy"
Sep 06 05:16:34.394761 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.394437    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" containerName="istio-proxy"
Sep 06 05:16:34.398238 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.398184    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"
Sep 06 05:16:34.400801 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.400778    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-494740-dockercfg-45jm7\""
Sep 06 05:16:34.400801 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.400791    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-494740\""
Sep 06 05:16:34.405986 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.405910    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"]
Sep 06 05:16:34.462287 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.462245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a65b31e0-ed0c-4e66-913a-1f38b47c37db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c\" (UID: \"a65b31e0-ed0c-4e66-913a-1f38b47c37db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"
Sep 06 05:16:34.462492 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.462391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvz48\" (UniqueName: \"kubernetes.io/projected/a65b31e0-ed0c-4e66-913a-1f38b47c37db-kube-api-access-cvz48\") pod \"payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c\" (UID: \"a65b31e0-ed0c-4e66-913a-1f38b47c37db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"
Sep 06 05:16:34.513162 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.513122    2578 generic.go:356] "Generic (PLEG): container finished" podID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" containerID="f58c3b85566824b52ed68806973f645798c7319a343378559e8e6c886f796068" exitCode=0
Sep 06 05:16:34.513335 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.513172    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"
Sep 06 05:16:34.513335 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.513201    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm" event={"ID":"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312","Type":"ContainerDied","Data":"f58c3b85566824b52ed68806973f645798c7319a343378559e8e6c886f796068"}
Sep 06 05:16:34.513335 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.513256    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm" event={"ID":"7e6a0a4d-5526-4e50-8fee-49ef9d2d1312","Type":"ContainerDied","Data":"2821b07898ce1c95166d30cecb754ed9f9653bce1c59b16913959964473582a5"}
Sep 06 05:16:34.513335 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.513284    2578 scope.go:117] "RemoveContainer" containerID="f58c3b85566824b52ed68806973f645798c7319a343378559e8e6c886f796068"
Sep 06 05:16:34.515480 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.515440    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58" event={"ID":"43725b3f-a6f4-4f8d-b614-679e019d3cb8","Type":"ContainerStarted","Data":"728c1779e7bd4b10d9d2ce95b824e7b90444d29c57338b976f238849346e79fe"}
Sep 06 05:16:34.515918 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.515898    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:34.524294 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.524248    2578 scope.go:117] "RemoveContainer" containerID="ab710e099ae8b8a7a8bfd39304f475903b9879839bf0846073d92e6f048e6727"
Sep 06 05:16:34.532535 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.532519    2578 scope.go:117] "RemoveContainer" containerID="f58c3b85566824b52ed68806973f645798c7319a343378559e8e6c886f796068"
Sep 06 05:16:34.532766 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:16:34.532749    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f58c3b85566824b52ed68806973f645798c7319a343378559e8e6c886f796068\": container with ID starting with f58c3b85566824b52ed68806973f645798c7319a343378559e8e6c886f796068 not found: ID does not exist" containerID="f58c3b85566824b52ed68806973f645798c7319a343378559e8e6c886f796068"
Sep 06 05:16:34.532811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.532774    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f58c3b85566824b52ed68806973f645798c7319a343378559e8e6c886f796068"} err="failed to get container status \"f58c3b85566824b52ed68806973f645798c7319a343378559e8e6c886f796068\": rpc error: code = NotFound desc = could not find container \"f58c3b85566824b52ed68806973f645798c7319a343378559e8e6c886f796068\": container with ID starting with f58c3b85566824b52ed68806973f645798c7319a343378559e8e6c886f796068 not found: ID does not exist"
Sep 06 05:16:34.532811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.532790    2578 scope.go:117] "RemoveContainer" containerID="ab710e099ae8b8a7a8bfd39304f475903b9879839bf0846073d92e6f048e6727"
Sep 06 05:16:34.533060 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:16:34.533045    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab710e099ae8b8a7a8bfd39304f475903b9879839bf0846073d92e6f048e6727\": container with ID starting with ab710e099ae8b8a7a8bfd39304f475903b9879839bf0846073d92e6f048e6727 not found: ID does not exist" containerID="ab710e099ae8b8a7a8bfd39304f475903b9879839bf0846073d92e6f048e6727"
Sep 06 05:16:34.533119 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.533066    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab710e099ae8b8a7a8bfd39304f475903b9879839bf0846073d92e6f048e6727"} err="failed to get container status \"ab710e099ae8b8a7a8bfd39304f475903b9879839bf0846073d92e6f048e6727\": rpc error: code = NotFound desc = could not find container \"ab710e099ae8b8a7a8bfd39304f475903b9879839bf0846073d92e6f048e6727\": container with ID starting with ab710e099ae8b8a7a8bfd39304f475903b9879839bf0846073d92e6f048e6727 not found: ID does not exist"
Sep 06 05:16:34.535750 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.535710    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58" podStartSLOduration=3.168803417 podStartE2EDuration="3.535696892s" totalImagesPullingTime="366.893475ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:16:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:16:34.534002058 +0000 UTC m=+1261.903457304" watchObservedRunningTime="2026-09-06 05:16:34.535696892 +0000 UTC m=+1261.905152138"
Sep 06 05:16:34.552203 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.552167    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"]
Sep 06 05:16:34.555482 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.555461    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-9035bc-openshift-default-884d9cddd-tgjqm"]
Sep 06 05:16:34.563896 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.563871    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a65b31e0-ed0c-4e66-913a-1f38b47c37db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c\" (UID: \"a65b31e0-ed0c-4e66-913a-1f38b47c37db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"
Sep 06 05:16:34.564277 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.564063    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvz48\" (UniqueName: \"kubernetes.io/projected/a65b31e0-ed0c-4e66-913a-1f38b47c37db-kube-api-access-cvz48\") pod \"payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c\" (UID: \"a65b31e0-ed0c-4e66-913a-1f38b47c37db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"
Sep 06 05:16:34.564686 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.564665    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a65b31e0-ed0c-4e66-913a-1f38b47c37db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c\" (UID: \"a65b31e0-ed0c-4e66-913a-1f38b47c37db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"
Sep 06 05:16:34.571473 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.571452    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvz48\" (UniqueName: \"kubernetes.io/projected/a65b31e0-ed0c-4e66-913a-1f38b47c37db-kube-api-access-cvz48\") pod \"payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c\" (UID: \"a65b31e0-ed0c-4e66-913a-1f38b47c37db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"
Sep 06 05:16:34.712334 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.712239    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"
Sep 06 05:16:34.982812 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:34.982735    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"]
Sep 06 05:16:34.986185 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:16:34.986149    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda65b31e0_ed0c_4e66_913a_1f38b47c37db.slice/crio-6c3177542d57ad726ee05305cc493139d74a173e7c33109dc60b776fcc2a963b WatchSource:0}: Error finding container 6c3177542d57ad726ee05305cc493139d74a173e7c33109dc60b776fcc2a963b: Status 404 returned error can't find the container with id 6c3177542d57ad726ee05305cc493139d74a173e7c33109dc60b776fcc2a963b
Sep 06 05:16:35.154850 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:35.154819    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" path="/var/lib/kubelet/pods/7e6a0a4d-5526-4e50-8fee-49ef9d2d1312/volumes"
Sep 06 05:16:35.245144 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:35.245063    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:35.523598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:35.523504    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c" event={"ID":"a65b31e0-ed0c-4e66-913a-1f38b47c37db","Type":"ContainerStarted","Data":"e4bd606c9a12d1493ce3a393a19dee3d6f4b55d51a0ae89069c7d222ee58fb09"}
Sep 06 05:16:35.523756 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:35.523608    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c" event={"ID":"a65b31e0-ed0c-4e66-913a-1f38b47c37db","Type":"ContainerStarted","Data":"6c3177542d57ad726ee05305cc493139d74a173e7c33109dc60b776fcc2a963b"}
Sep 06 05:16:35.523756 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:35.523749    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"
Sep 06 05:16:35.538587 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:35.538541    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c" podStartSLOduration=1.538526807 podStartE2EDuration="1.538526807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:16:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:16:35.537550885 +0000 UTC m=+1262.907006132" watchObservedRunningTime="2026-09-06 05:16:35.538526807 +0000 UTC m=+1262.907982053"
Sep 06 05:16:39.433970 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:39.433937    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:40.530121 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:40.530095    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:16:44.518790 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:44.518756    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"
Sep 06 05:16:46.531349 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:46.531317    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"
Sep 06 05:16:47.803786 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:47.803096    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": read tcp 10.133.0.2:57486->10.133.0.66:15021: read: connection reset by peer" start-of-body=
Sep 06 05:16:47.803786 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:47.803168    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq" podUID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": read tcp 10.133.0.2:57486->10.133.0.66:15021: read: connection reset by peer"
Sep 06 05:16:48.604852 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:48.604819    2578 generic.go:356] "Generic (PLEG): container finished" podID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerID="f51a1464a91f98bc0f9db712a36d9182343e51340b1527584fc339ccd4146f33" exitCode=0
Sep 06 05:16:48.605070 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:48.604894    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq" event={"ID":"86d5a085-cde4-4d06-ab3d-65f550224a3e","Type":"ContainerDied","Data":"f51a1464a91f98bc0f9db712a36d9182343e51340b1527584fc339ccd4146f33"}
Sep 06 05:16:48.605404 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:48.605388    2578 scope.go:117] "RemoveContainer" containerID="f51a1464a91f98bc0f9db712a36d9182343e51340b1527584fc339ccd4146f33"
Sep 06 05:16:48.606439 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:48.606406    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:16:48.606499 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:48.606481    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:16:48.606532 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:48.606514    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:16:49.616491 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:49.616440    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq" event={"ID":"86d5a085-cde4-4d06-ab3d-65f550224a3e","Type":"ContainerStarted","Data":"6a0b642357180461692905f2e3bcc14008d3ced8f0e740cb7fe73952257ac82b"}
Sep 06 05:16:49.845536 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:49.845485    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:49.845945 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:49.845921    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 06 05:16:49.846103 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:49.845977    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq" podUID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 06 05:16:50.845421 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:50.845382    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 06 05:16:50.845811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:50.845436    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq" podUID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 06 05:16:51.845763 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:51.845723    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 06 05:16:51.846339 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:51.846310    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq" podUID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 06 05:16:52.845169 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:52.845134    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 06 05:16:52.845339 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:52.845200    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq" podUID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 06 05:16:52.985263 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:52.985229    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:53.845616 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:53.845574    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 06 05:16:53.845800 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:53.845639    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq" podUID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 06 05:16:54.196042 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:54.195978    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:54.248299 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:54.248273    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:54.660070 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:54.660007    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:54.848663 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:54.848627    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:54.848901 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:54.848883    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:54.849671 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:54.849649    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:16:55.786576 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:55.786540    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:57.280690 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:57.280655    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:57.586926 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:57.586839    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:57.892864 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:57.892823    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:16:58.027938 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:16:58.027905    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:01.065089 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:01.065055    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:01.162661 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:01.162632    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:02.220938 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.220896    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:02.318774 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.318731    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:02.470643 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.470603    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:02.678710 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.678679    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:02.789538 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.789493    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:02.803619 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.803587    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"]
Sep 06 05:17:02.804006 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.803993    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" containerName="istio-proxy"
Sep 06 05:17:02.804072 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.804007    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e6a0a4d-5526-4e50-8fee-49ef9d2d1312" containerName="istio-proxy"
Sep 06 05:17:02.807232 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.807205    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"
Sep 06 05:17:02.809621 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.809604    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-167b5b-dockercfg-5d7c9\""
Sep 06 05:17:02.809728 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.809663    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-167b5b\""
Sep 06 05:17:02.817348 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.817324    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"]
Sep 06 05:17:02.847840 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.847805    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmj6j\" (UniqueName: \"kubernetes.io/projected/e52ec6f2-7df0-488e-b19e-97887e27e36a-kube-api-access-nmj6j\") pod \"payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5\" (UID: \"e52ec6f2-7df0-488e-b19e-97887e27e36a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"
Sep 06 05:17:02.848038 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.847854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52ec6f2-7df0-488e-b19e-97887e27e36a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5\" (UID: \"e52ec6f2-7df0-488e-b19e-97887e27e36a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"
Sep 06 05:17:02.948693 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.948609    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmj6j\" (UniqueName: \"kubernetes.io/projected/e52ec6f2-7df0-488e-b19e-97887e27e36a-kube-api-access-nmj6j\") pod \"payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5\" (UID: \"e52ec6f2-7df0-488e-b19e-97887e27e36a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"
Sep 06 05:17:02.948693 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.948659    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52ec6f2-7df0-488e-b19e-97887e27e36a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5\" (UID: \"e52ec6f2-7df0-488e-b19e-97887e27e36a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"
Sep 06 05:17:02.949439 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.949413    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52ec6f2-7df0-488e-b19e-97887e27e36a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5\" (UID: \"e52ec6f2-7df0-488e-b19e-97887e27e36a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"
Sep 06 05:17:02.956438 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:02.956410    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmj6j\" (UniqueName: \"kubernetes.io/projected/e52ec6f2-7df0-488e-b19e-97887e27e36a-kube-api-access-nmj6j\") pod \"payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5\" (UID: \"e52ec6f2-7df0-488e-b19e-97887e27e36a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"
Sep 06 05:17:03.118988 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:03.118944    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"
Sep 06 05:17:03.395580 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:03.395550    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"]
Sep 06 05:17:03.398496 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:17:03.398465    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode52ec6f2_7df0_488e_b19e_97887e27e36a.slice/crio-8f1accb66ae10d454ca61d7bd36be9a587262926648e1f4e4de24a97050ea499 WatchSource:0}: Error finding container 8f1accb66ae10d454ca61d7bd36be9a587262926648e1f4e4de24a97050ea499: Status 404 returned error can't find the container with id 8f1accb66ae10d454ca61d7bd36be9a587262926648e1f4e4de24a97050ea499
Sep 06 05:17:03.694447 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:03.694352    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5" event={"ID":"e52ec6f2-7df0-488e-b19e-97887e27e36a","Type":"ContainerStarted","Data":"6a44672b91f44362f7ee49259f5fb970344248ce40a8558c698b6c4cc9a2dcf0"}
Sep 06 05:17:03.694447 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:03.694424    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5" event={"ID":"e52ec6f2-7df0-488e-b19e-97887e27e36a","Type":"ContainerStarted","Data":"8f1accb66ae10d454ca61d7bd36be9a587262926648e1f4e4de24a97050ea499"}
Sep 06 05:17:03.694671 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:03.694466    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"
Sep 06 05:17:03.709773 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:03.709708    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5" podStartSLOduration=1.709686911 podStartE2EDuration="1.709686911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:17:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:17:03.70884221 +0000 UTC m=+1291.078297458" watchObservedRunningTime="2026-09-06 05:17:03.709686911 +0000 UTC m=+1291.079142160"
Sep 06 05:17:09.504403 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:09.504364    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:09.801589 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:09.801486    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:09.900868 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:09.900834    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:13.171604 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:13.171539    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:13.271778 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:13.271739    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:14.439959 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:14.439904    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:14.539943 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:14.539909    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:14.702323 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:14.702239    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"
Sep 06 05:17:14.886400 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:14.886360    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:14.985489 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:14.985391    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:14.986531 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:14.986511    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"]
Sep 06 05:17:14.991515 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:14.991492    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"
Sep 06 05:17:14.993875 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:14.993854    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-1b19002b\""
Sep 06 05:17:14.994004 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:14.993951    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-1b19002b-dockercfg-5p55b\""
Sep 06 05:17:15.005719 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.005693    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"]
Sep 06 05:17:15.021370 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.021341    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"]
Sep 06 05:17:15.024939 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.024918    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"
Sep 06 05:17:15.033793 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.033758    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"]
Sep 06 05:17:15.062963 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.062928    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2458g\" (UniqueName: \"kubernetes.io/projected/7e0582d7-be0b-460c-8db5-7a0dd7226b0a-kube-api-access-2458g\") pod \"payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5\" (UID: \"7e0582d7-be0b-460c-8db5-7a0dd7226b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"
Sep 06 05:17:15.063146 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.063117    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e0582d7-be0b-460c-8db5-7a0dd7226b0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5\" (UID: \"7e0582d7-be0b-460c-8db5-7a0dd7226b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"
Sep 06 05:17:15.164679 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.164643    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2458g\" (UniqueName: \"kubernetes.io/projected/7e0582d7-be0b-460c-8db5-7a0dd7226b0a-kube-api-access-2458g\") pod \"payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5\" (UID: \"7e0582d7-be0b-460c-8db5-7a0dd7226b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"
Sep 06 05:17:15.164849 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.164768    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl26w\" (UniqueName: \"kubernetes.io/projected/a200a74e-8e02-4fae-9f79-369ab599be98-kube-api-access-kl26w\") pod \"payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j\" (UID: \"a200a74e-8e02-4fae-9f79-369ab599be98\") " pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"
Sep 06 05:17:15.164849 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.164809    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a200a74e-8e02-4fae-9f79-369ab599be98-plugins-config-volume\") pod \"payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j\" (UID: \"a200a74e-8e02-4fae-9f79-369ab599be98\") " pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"
Sep 06 05:17:15.164954 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.164924    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e0582d7-be0b-460c-8db5-7a0dd7226b0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5\" (UID: \"7e0582d7-be0b-460c-8db5-7a0dd7226b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"
Sep 06 05:17:15.165580 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.165562    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e0582d7-be0b-460c-8db5-7a0dd7226b0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5\" (UID: \"7e0582d7-be0b-460c-8db5-7a0dd7226b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"
Sep 06 05:17:15.173066 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.172999    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2458g\" (UniqueName: \"kubernetes.io/projected/7e0582d7-be0b-460c-8db5-7a0dd7226b0a-kube-api-access-2458g\") pod \"payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5\" (UID: \"7e0582d7-be0b-460c-8db5-7a0dd7226b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"
Sep 06 05:17:15.266558 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.266478    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl26w\" (UniqueName: \"kubernetes.io/projected/a200a74e-8e02-4fae-9f79-369ab599be98-kube-api-access-kl26w\") pod \"payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j\" (UID: \"a200a74e-8e02-4fae-9f79-369ab599be98\") " pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"
Sep 06 05:17:15.266558 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.266522    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a200a74e-8e02-4fae-9f79-369ab599be98-plugins-config-volume\") pod \"payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j\" (UID: \"a200a74e-8e02-4fae-9f79-369ab599be98\") " pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"
Sep 06 05:17:15.267235 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.267216    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a200a74e-8e02-4fae-9f79-369ab599be98-plugins-config-volume\") pod \"payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j\" (UID: \"a200a74e-8e02-4fae-9f79-369ab599be98\") " pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"
Sep 06 05:17:15.274042 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.274003    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl26w\" (UniqueName: \"kubernetes.io/projected/a200a74e-8e02-4fae-9f79-369ab599be98-kube-api-access-kl26w\") pod \"payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j\" (UID: \"a200a74e-8e02-4fae-9f79-369ab599be98\") " pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"
Sep 06 05:17:15.302524 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.302488    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"
Sep 06 05:17:15.338602 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.338569    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"
Sep 06 05:17:15.569520 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.569434    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:15.575921 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.575896    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"]
Sep 06 05:17:15.577857 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:17:15.577834    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e0582d7_be0b_460c_8db5_7a0dd7226b0a.slice/crio-0a5f2773140495a96465ac944586f3eeea088ff7a7603708f17f9e38a8d39359 WatchSource:0}: Error finding container 0a5f2773140495a96465ac944586f3eeea088ff7a7603708f17f9e38a8d39359: Status 404 returned error can't find the container with id 0a5f2773140495a96465ac944586f3eeea088ff7a7603708f17f9e38a8d39359
Sep 06 05:17:15.631573 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.631549    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"]
Sep 06 05:17:15.633627 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:17:15.633597    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda200a74e_8e02_4fae_9f79_369ab599be98.slice/crio-6945a348fc8d46fd4fe19317ac9f38c0cdd8977848e7212fc4711cc6a6dfad1f WatchSource:0}: Error finding container 6945a348fc8d46fd4fe19317ac9f38c0cdd8977848e7212fc4711cc6a6dfad1f: Status 404 returned error can't find the container with id 6945a348fc8d46fd4fe19317ac9f38c0cdd8977848e7212fc4711cc6a6dfad1f
Sep 06 05:17:15.764933 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.764896    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5" event={"ID":"7e0582d7-be0b-460c-8db5-7a0dd7226b0a","Type":"ContainerStarted","Data":"30788ec9d7cc65e26523df948887744f2e9322d819fa6fa8b0104e6f1982f34a"}
Sep 06 05:17:15.765149 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.765068    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"
Sep 06 05:17:15.765149 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.765116    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5" event={"ID":"7e0582d7-be0b-460c-8db5-7a0dd7226b0a","Type":"ContainerStarted","Data":"0a5f2773140495a96465ac944586f3eeea088ff7a7603708f17f9e38a8d39359"}
Sep 06 05:17:15.766681 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.766659    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j" event={"ID":"a200a74e-8e02-4fae-9f79-369ab599be98","Type":"ContainerStarted","Data":"ced08a672c1be96a59928531f0cdc956d0a6f2b26169900887f10171205762b5"}
Sep 06 05:17:15.766820 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.766700    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"
Sep 06 05:17:15.766820 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.766715    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j" event={"ID":"a200a74e-8e02-4fae-9f79-369ab599be98","Type":"ContainerStarted","Data":"6945a348fc8d46fd4fe19317ac9f38c0cdd8977848e7212fc4711cc6a6dfad1f"}
Sep 06 05:17:15.781675 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.781617    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5" podStartSLOduration=1.781601868 podStartE2EDuration="1.781601868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:17:15.77967368 +0000 UTC m=+1303.149128927" watchObservedRunningTime="2026-09-06 05:17:15.781601868 +0000 UTC m=+1303.151057113"
Sep 06 05:17:15.793193 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:15.793139    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j" podStartSLOduration=1.7931239749999999 podStartE2EDuration="1.793123975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:17:15.792890111 +0000 UTC m=+1303.162345361" watchObservedRunningTime="2026-09-06 05:17:15.793123975 +0000 UTC m=+1303.162579222"
Sep 06 05:17:23.135324 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:23.135284    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:26.020392 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:26.020352    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:26.774227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:26.774183    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"
Sep 06 05:17:26.774439 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:26.774267    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"
Sep 06 05:17:31.072811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:31.072774    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:35.346182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:35.346147    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:38.108843 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:38.108810    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:48.876784 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:48.876751    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:55.261661 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:55.261623    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:59.543407 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.543370    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"]
Sep 06 05:17:59.549274 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.549245    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.552443 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.552413    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-167b5b\"/\"openshift-service-ca.crt\""
Sep 06 05:17:59.552579 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.552474    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-167b5b\"/\"default-dockercfg-v2ch9\""
Sep 06 05:17:59.552579 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.552535    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-167b5b\"/\"kube-root-ca.crt\""
Sep 06 05:17:59.552703 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.552586    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-167b5b\"/\"auth-test-model-167b5b-kserve-self-signed-certs\""
Sep 06 05:17:59.558193 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.558171    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"]
Sep 06 05:17:59.568642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.568610    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:17:59.717413 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.717380    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-tmp-dir\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.717589 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.717426    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97da5596-6423-4220-99af-5fcc199879cf-tls-certs\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.717641 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.717581    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77b7s\" (UniqueName: \"kubernetes.io/projected/97da5596-6423-4220-99af-5fcc199879cf-kube-api-access-77b7s\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.717641 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.717629    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-model-cache\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.717720 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.717662    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-dshm\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.717720 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.717703    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-home\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.818744 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.818652    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-tmp-dir\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.818744 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.818701    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97da5596-6423-4220-99af-5fcc199879cf-tls-certs\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.818959 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.818776    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77b7s\" (UniqueName: \"kubernetes.io/projected/97da5596-6423-4220-99af-5fcc199879cf-kube-api-access-77b7s\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.818959 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.818816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-model-cache\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.819072 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.818988    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-dshm\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.819128 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.819064    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-home\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.819173 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.819134    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-tmp-dir\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.819243 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.819184    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-model-cache\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.819482 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.819459    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-home\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.821136 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.821117    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-dshm\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.821293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.821277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97da5596-6423-4220-99af-5fcc199879cf-tls-certs\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.826204 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.826182    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77b7s\" (UniqueName: \"kubernetes.io/projected/97da5596-6423-4220-99af-5fcc199879cf-kube-api-access-77b7s\") pod \"auth-test-model-167b5b-kserve-6c5474f96c-5sddk\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") " pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:17:59.860402 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:17:59.860372    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:18:00.130412 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:00.130383    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"]
Sep 06 05:18:00.132776 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:18:00.132745    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97da5596_6423_4220_99af_5fcc199879cf.slice/crio-ec299ead4015a5269b87bd3edc5bbc8e51e70d2699688bf6f3baf636a6aadb91 WatchSource:0}: Error finding container ec299ead4015a5269b87bd3edc5bbc8e51e70d2699688bf6f3baf636a6aadb91: Status 404 returned error can't find the container with id ec299ead4015a5269b87bd3edc5bbc8e51e70d2699688bf6f3baf636a6aadb91
Sep 06 05:18:01.019296 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:01.019256    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk" event={"ID":"97da5596-6423-4220-99af-5fcc199879cf","Type":"ContainerStarted","Data":"d4c240c027ca3f0a08c674ff0512ec8efb22b581bf4c2cf292e873b806ff52e5"}
Sep 06 05:18:01.019296 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:01.019296    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk" event={"ID":"97da5596-6423-4220-99af-5fcc199879cf","Type":"ContainerStarted","Data":"ec299ead4015a5269b87bd3edc5bbc8e51e70d2699688bf6f3baf636a6aadb91"}
Sep 06 05:18:01.035596 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:01.035545    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk" podStartSLOduration=2.035527987 podStartE2EDuration="2.035527987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:17:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:18:01.033220427 +0000 UTC m=+1348.402675673" watchObservedRunningTime="2026-09-06 05:18:01.035527987 +0000 UTC m=+1348.404983232"
Sep 06 05:18:06.654593 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:06.654549    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:18:06.755657 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:06.755622    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:18:09.860550 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:09.860514    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:18:09.861060 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:09.860574    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:18:09.872729 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:09.872700    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:18:10.079793 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:10.079764    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:18:29.310568 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:29.310533    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:18:29.411824 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:29.411784    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:18:33.275879 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:33.275841    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:18:48.668920 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:48.668884    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:18:48.693765 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:48.693731    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"]
Sep 06 05:18:48.694246 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:48.694211    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j" podUID="a200a74e-8e02-4fae-9f79-369ab599be98" containerName="payload-processing" containerID="cri-o://ced08a672c1be96a59928531f0cdc956d0a6f2b26169900887f10171205762b5" gracePeriod=30
Sep 06 05:18:48.697331 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:48.697308    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"]
Sep 06 05:18:48.697589 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:48.697542    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5" podUID="7e0582d7-be0b-460c-8db5-7a0dd7226b0a" containerName="payload-pre-processing" containerID="cri-o://30788ec9d7cc65e26523df948887744f2e9322d819fa6fa8b0104e6f1982f34a" gracePeriod=30
Sep 06 05:18:48.730312 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:48.730272    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:18:48.786307 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:48.786272    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:18:48.852207 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:48.852173    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:18:49.006095 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.006067    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"
Sep 06 05:18:49.009337 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.009314    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"
Sep 06 05:18:49.109515 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.109482    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e0582d7-be0b-460c-8db5-7a0dd7226b0a-plugins-config-volume\") pod \"7e0582d7-be0b-460c-8db5-7a0dd7226b0a\" (UID: \"7e0582d7-be0b-460c-8db5-7a0dd7226b0a\") "
Sep 06 05:18:49.109660 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.109542    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kl26w\" (UniqueName: \"kubernetes.io/projected/a200a74e-8e02-4fae-9f79-369ab599be98-kube-api-access-kl26w\") pod \"a200a74e-8e02-4fae-9f79-369ab599be98\" (UID: \"a200a74e-8e02-4fae-9f79-369ab599be98\") "
Sep 06 05:18:49.109660 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.109583    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a200a74e-8e02-4fae-9f79-369ab599be98-plugins-config-volume\") pod \"a200a74e-8e02-4fae-9f79-369ab599be98\" (UID: \"a200a74e-8e02-4fae-9f79-369ab599be98\") "
Sep 06 05:18:49.109745 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.109674    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2458g\" (UniqueName: \"kubernetes.io/projected/7e0582d7-be0b-460c-8db5-7a0dd7226b0a-kube-api-access-2458g\") pod \"7e0582d7-be0b-460c-8db5-7a0dd7226b0a\" (UID: \"7e0582d7-be0b-460c-8db5-7a0dd7226b0a\") "
Sep 06 05:18:49.109877 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.109850    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e0582d7-be0b-460c-8db5-7a0dd7226b0a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7e0582d7-be0b-460c-8db5-7a0dd7226b0a" (UID: "7e0582d7-be0b-460c-8db5-7a0dd7226b0a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:18:49.110039 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.110002    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e0582d7-be0b-460c-8db5-7a0dd7226b0a-plugins-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:18:49.110167 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.110084    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a200a74e-8e02-4fae-9f79-369ab599be98-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a200a74e-8e02-4fae-9f79-369ab599be98" (UID: "a200a74e-8e02-4fae-9f79-369ab599be98"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:18:49.111652 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.111631    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a200a74e-8e02-4fae-9f79-369ab599be98-kube-api-access-kl26w" (OuterVolumeSpecName: "kube-api-access-kl26w") pod "a200a74e-8e02-4fae-9f79-369ab599be98" (UID: "a200a74e-8e02-4fae-9f79-369ab599be98"). InnerVolumeSpecName "kube-api-access-kl26w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:18:49.111652 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.111643    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e0582d7-be0b-460c-8db5-7a0dd7226b0a-kube-api-access-2458g" (OuterVolumeSpecName: "kube-api-access-2458g") pod "7e0582d7-be0b-460c-8db5-7a0dd7226b0a" (UID: "7e0582d7-be0b-460c-8db5-7a0dd7226b0a"). InnerVolumeSpecName "kube-api-access-2458g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:18:49.210832 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.210796    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl26w\" (UniqueName: \"kubernetes.io/projected/a200a74e-8e02-4fae-9f79-369ab599be98-kube-api-access-kl26w\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:18:49.210832 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.210829    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a200a74e-8e02-4fae-9f79-369ab599be98-plugins-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:18:49.211042 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.210843    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2458g\" (UniqueName: \"kubernetes.io/projected/7e0582d7-be0b-460c-8db5-7a0dd7226b0a-kube-api-access-2458g\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:18:49.287047 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.286945    2578 generic.go:356] "Generic (PLEG): container finished" podID="7e0582d7-be0b-460c-8db5-7a0dd7226b0a" containerID="30788ec9d7cc65e26523df948887744f2e9322d819fa6fa8b0104e6f1982f34a" exitCode=0
Sep 06 05:18:49.287047 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.286974    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5" event={"ID":"7e0582d7-be0b-460c-8db5-7a0dd7226b0a","Type":"ContainerDied","Data":"30788ec9d7cc65e26523df948887744f2e9322d819fa6fa8b0104e6f1982f34a"}
Sep 06 05:18:49.287047 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.286990    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"
Sep 06 05:18:49.287329 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.287069    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5" event={"ID":"7e0582d7-be0b-460c-8db5-7a0dd7226b0a","Type":"ContainerDied","Data":"0a5f2773140495a96465ac944586f3eeea088ff7a7603708f17f9e38a8d39359"}
Sep 06 05:18:49.287329 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.287103    2578 scope.go:117] "RemoveContainer" containerID="30788ec9d7cc65e26523df948887744f2e9322d819fa6fa8b0104e6f1982f34a"
Sep 06 05:18:49.288660 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.288635    2578 generic.go:356] "Generic (PLEG): container finished" podID="a200a74e-8e02-4fae-9f79-369ab599be98" containerID="ced08a672c1be96a59928531f0cdc956d0a6f2b26169900887f10171205762b5" exitCode=0
Sep 06 05:18:49.288791 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.288697    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"
Sep 06 05:18:49.288791 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.288748    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j" event={"ID":"a200a74e-8e02-4fae-9f79-369ab599be98","Type":"ContainerDied","Data":"ced08a672c1be96a59928531f0cdc956d0a6f2b26169900887f10171205762b5"}
Sep 06 05:18:49.288791 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.288789    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j" event={"ID":"a200a74e-8e02-4fae-9f79-369ab599be98","Type":"ContainerDied","Data":"6945a348fc8d46fd4fe19317ac9f38c0cdd8977848e7212fc4711cc6a6dfad1f"}
Sep 06 05:18:49.297173 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.297152    2578 scope.go:117] "RemoveContainer" containerID="30788ec9d7cc65e26523df948887744f2e9322d819fa6fa8b0104e6f1982f34a"
Sep 06 05:18:49.297455 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:18:49.297433    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30788ec9d7cc65e26523df948887744f2e9322d819fa6fa8b0104e6f1982f34a\": container with ID starting with 30788ec9d7cc65e26523df948887744f2e9322d819fa6fa8b0104e6f1982f34a not found: ID does not exist" containerID="30788ec9d7cc65e26523df948887744f2e9322d819fa6fa8b0104e6f1982f34a"
Sep 06 05:18:49.297534 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.297465    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30788ec9d7cc65e26523df948887744f2e9322d819fa6fa8b0104e6f1982f34a"} err="failed to get container status \"30788ec9d7cc65e26523df948887744f2e9322d819fa6fa8b0104e6f1982f34a\": rpc error: code = NotFound desc = could not find container \"30788ec9d7cc65e26523df948887744f2e9322d819fa6fa8b0104e6f1982f34a\": container with ID starting with 30788ec9d7cc65e26523df948887744f2e9322d819fa6fa8b0104e6f1982f34a not found: ID does not exist"
Sep 06 05:18:49.297534 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.297482    2578 scope.go:117] "RemoveContainer" containerID="ced08a672c1be96a59928531f0cdc956d0a6f2b26169900887f10171205762b5"
Sep 06 05:18:49.304600 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.304578    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"]
Sep 06 05:18:49.307123 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.307093    2578 scope.go:117] "RemoveContainer" containerID="ced08a672c1be96a59928531f0cdc956d0a6f2b26169900887f10171205762b5"
Sep 06 05:18:49.307441 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:18:49.307419    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ced08a672c1be96a59928531f0cdc956d0a6f2b26169900887f10171205762b5\": container with ID starting with ced08a672c1be96a59928531f0cdc956d0a6f2b26169900887f10171205762b5 not found: ID does not exist" containerID="ced08a672c1be96a59928531f0cdc956d0a6f2b26169900887f10171205762b5"
Sep 06 05:18:49.307557 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.307450    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ced08a672c1be96a59928531f0cdc956d0a6f2b26169900887f10171205762b5"} err="failed to get container status \"ced08a672c1be96a59928531f0cdc956d0a6f2b26169900887f10171205762b5\": rpc error: code = NotFound desc = could not find container \"ced08a672c1be96a59928531f0cdc956d0a6f2b26169900887f10171205762b5\": container with ID starting with ced08a672c1be96a59928531f0cdc956d0a6f2b26169900887f10171205762b5 not found: ID does not exist"
Sep 06 05:18:49.308172 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.308153    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1b19002b-b94d4dd98-mlzm5"]
Sep 06 05:18:49.316784 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.316762    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"]
Sep 06 05:18:49.319642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:49.319621    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1b19002b-6d78d98cbf-4j97j"]
Sep 06 05:18:51.154836 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:51.154794    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e0582d7-be0b-460c-8db5-7a0dd7226b0a" path="/var/lib/kubelet/pods/7e0582d7-be0b-460c-8db5-7a0dd7226b0a/volumes"
Sep 06 05:18:51.155398 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:51.155377    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a200a74e-8e02-4fae-9f79-369ab599be98" path="/var/lib/kubelet/pods/a200a74e-8e02-4fae-9f79-369ab599be98/volumes"
Sep 06 05:18:51.966363 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:51.966320    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:18:56.994373 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:18:56.994333    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:19:34.055114 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:19:34.055057    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:19:34.105067 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:19:34.105007    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:19:39.327872 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:19:39.327838    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:19:44.129730 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:19:44.129688    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:19:44.177276 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:19:44.177244    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:01.554334 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:01.554284    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:01.627865 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:01.627831    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:01.875207 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:01.875175    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:03.171122 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:03.171084    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:03.270120 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:03.270089    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:03.370752 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:03.370694    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:04.043531 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:04.043494    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:04.146371 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:04.146331    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:04.243971 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:04.243936    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:05.522866 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:05.522823    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:11.243379 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:11.243342    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:11.343642 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:11.343596    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:11.913548 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:11.913514    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:15.353799 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:15.353756    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:16.248646 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:16.248602    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"]
Sep 06 05:20:16.260159 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:16.260112    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:17.617258 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:17.617215    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:17.677059 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:17.677010    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:17.757459 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:17.757423    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:17.865828 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:17.865786    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:22.461787 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:22.461741    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:28.345203 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:28.345162    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:29.449443 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:29.449403    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:31.268832 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.268791    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk" podUID="97da5596-6423-4220-99af-5fcc199879cf" containerName="main" containerID="cri-o://d4c240c027ca3f0a08c674ff0512ec8efb22b581bf4c2cf292e873b806ff52e5" gracePeriod=45
Sep 06 05:20:31.527785 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.527729    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:20:31.643801 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.643768    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-home\") pod \"97da5596-6423-4220-99af-5fcc199879cf\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") "
Sep 06 05:20:31.644006 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.643830    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97da5596-6423-4220-99af-5fcc199879cf-tls-certs\") pod \"97da5596-6423-4220-99af-5fcc199879cf\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") "
Sep 06 05:20:31.644006 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.643879    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77b7s\" (UniqueName: \"kubernetes.io/projected/97da5596-6423-4220-99af-5fcc199879cf-kube-api-access-77b7s\") pod \"97da5596-6423-4220-99af-5fcc199879cf\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") "
Sep 06 05:20:31.644006 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.643908    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-dshm\") pod \"97da5596-6423-4220-99af-5fcc199879cf\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") "
Sep 06 05:20:31.644006 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.643993    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-tmp-dir\") pod \"97da5596-6423-4220-99af-5fcc199879cf\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") "
Sep 06 05:20:31.644263 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.644097    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-model-cache\") pod \"97da5596-6423-4220-99af-5fcc199879cf\" (UID: \"97da5596-6423-4220-99af-5fcc199879cf\") "
Sep 06 05:20:31.644263 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.644101    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-home" (OuterVolumeSpecName: "home") pod "97da5596-6423-4220-99af-5fcc199879cf" (UID: "97da5596-6423-4220-99af-5fcc199879cf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:20:31.644406 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.644383    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-model-cache" (OuterVolumeSpecName: "model-cache") pod "97da5596-6423-4220-99af-5fcc199879cf" (UID: "97da5596-6423-4220-99af-5fcc199879cf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:20:31.644469 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.644383    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "97da5596-6423-4220-99af-5fcc199879cf" (UID: "97da5596-6423-4220-99af-5fcc199879cf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:20:31.644594 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.644570    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-tmp-dir\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:20:31.644594 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.644597    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-model-cache\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:20:31.644745 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.644611    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-home\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:20:31.646245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.646215    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-dshm" (OuterVolumeSpecName: "dshm") pod "97da5596-6423-4220-99af-5fcc199879cf" (UID: "97da5596-6423-4220-99af-5fcc199879cf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:20:31.646338 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.646238    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97da5596-6423-4220-99af-5fcc199879cf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "97da5596-6423-4220-99af-5fcc199879cf" (UID: "97da5596-6423-4220-99af-5fcc199879cf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:20:31.646338 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.646219    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97da5596-6423-4220-99af-5fcc199879cf-kube-api-access-77b7s" (OuterVolumeSpecName: "kube-api-access-77b7s") pod "97da5596-6423-4220-99af-5fcc199879cf" (UID: "97da5596-6423-4220-99af-5fcc199879cf"). InnerVolumeSpecName "kube-api-access-77b7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:20:31.746215 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.746179    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97da5596-6423-4220-99af-5fcc199879cf-tls-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:20:31.746215 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.746211    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77b7s\" (UniqueName: \"kubernetes.io/projected/97da5596-6423-4220-99af-5fcc199879cf-kube-api-access-77b7s\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:20:31.746215 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.746221    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97da5596-6423-4220-99af-5fcc199879cf-dshm\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:20:31.850755 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.850668    2578 generic.go:356] "Generic (PLEG): container finished" podID="97da5596-6423-4220-99af-5fcc199879cf" containerID="d4c240c027ca3f0a08c674ff0512ec8efb22b581bf4c2cf292e873b806ff52e5" exitCode=0
Sep 06 05:20:31.850755 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.850708    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk" event={"ID":"97da5596-6423-4220-99af-5fcc199879cf","Type":"ContainerDied","Data":"d4c240c027ca3f0a08c674ff0512ec8efb22b581bf4c2cf292e873b806ff52e5"}
Sep 06 05:20:31.850755 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.850714    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"
Sep 06 05:20:31.850755 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.850733    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk" event={"ID":"97da5596-6423-4220-99af-5fcc199879cf","Type":"ContainerDied","Data":"ec299ead4015a5269b87bd3edc5bbc8e51e70d2699688bf6f3baf636a6aadb91"}
Sep 06 05:20:31.850755 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.850753    2578 scope.go:117] "RemoveContainer" containerID="d4c240c027ca3f0a08c674ff0512ec8efb22b581bf4c2cf292e873b806ff52e5"
Sep 06 05:20:31.860171 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.860153    2578 scope.go:117] "RemoveContainer" containerID="d4c240c027ca3f0a08c674ff0512ec8efb22b581bf4c2cf292e873b806ff52e5"
Sep 06 05:20:31.860427 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:20:31.860410    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4c240c027ca3f0a08c674ff0512ec8efb22b581bf4c2cf292e873b806ff52e5\": container with ID starting with d4c240c027ca3f0a08c674ff0512ec8efb22b581bf4c2cf292e873b806ff52e5 not found: ID does not exist" containerID="d4c240c027ca3f0a08c674ff0512ec8efb22b581bf4c2cf292e873b806ff52e5"
Sep 06 05:20:31.860494 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.860439    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4c240c027ca3f0a08c674ff0512ec8efb22b581bf4c2cf292e873b806ff52e5"} err="failed to get container status \"d4c240c027ca3f0a08c674ff0512ec8efb22b581bf4c2cf292e873b806ff52e5\": rpc error: code = NotFound desc = could not find container \"d4c240c027ca3f0a08c674ff0512ec8efb22b581bf4c2cf292e873b806ff52e5\": container with ID starting with d4c240c027ca3f0a08c674ff0512ec8efb22b581bf4c2cf292e873b806ff52e5 not found: ID does not exist"
Sep 06 05:20:31.871275 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.871254    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"]
Sep 06 05:20:31.875210 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:31.875188    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-167b5b/auth-test-model-167b5b-kserve-6c5474f96c-5sddk"]
Sep 06 05:20:33.154617 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:33.154588    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97da5596-6423-4220-99af-5fcc199879cf" path="/var/lib/kubelet/pods/97da5596-6423-4220-99af-5fcc199879cf/volumes"
Sep 06 05:20:49.487330 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:49.487283    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:55.205919 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:55.205881    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:55.263956 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:55.263924    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:55.541751 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:55.541640    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:55.836845 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:55.836734    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:55.956563 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:55.956528    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:58.465086 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:58.465039    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:58.562268 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:58.562233    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:59.615398 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:59.615361    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:20:59.713978 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:20:59.713946    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:00.053338 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.053296    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:00.150871 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.150840    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:00.224814 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.224784    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:00.232087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.232063    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"]
Sep 06 05:21:00.232467 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.232455    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e0582d7-be0b-460c-8db5-7a0dd7226b0a" containerName="payload-pre-processing"
Sep 06 05:21:00.232520 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.232469    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e0582d7-be0b-460c-8db5-7a0dd7226b0a" containerName="payload-pre-processing"
Sep 06 05:21:00.232520 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.232488    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a200a74e-8e02-4fae-9f79-369ab599be98" containerName="payload-processing"
Sep 06 05:21:00.232520 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.232494    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a200a74e-8e02-4fae-9f79-369ab599be98" containerName="payload-processing"
Sep 06 05:21:00.232520 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.232513    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97da5596-6423-4220-99af-5fcc199879cf" containerName="main"
Sep 06 05:21:00.232520 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.232518    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="97da5596-6423-4220-99af-5fcc199879cf" containerName="main"
Sep 06 05:21:00.232692 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.232584    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e0582d7-be0b-460c-8db5-7a0dd7226b0a" containerName="payload-pre-processing"
Sep 06 05:21:00.232692 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.232592    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a200a74e-8e02-4fae-9f79-369ab599be98" containerName="payload-processing"
Sep 06 05:21:00.232692 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.232601    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="97da5596-6423-4220-99af-5fcc199879cf" containerName="main"
Sep 06 05:21:00.235704 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.235686    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"
Sep 06 05:21:00.238047 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.238029    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-71403e-dockercfg-dbj6v\""
Sep 06 05:21:00.238142 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.238082    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-71403e\""
Sep 06 05:21:00.245520 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.245496    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"]
Sep 06 05:21:00.340951 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.340846    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2382715c-e2ff-4ef7-983f-0bebcbac420c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk\" (UID: \"2382715c-e2ff-4ef7-983f-0bebcbac420c\") " pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"
Sep 06 05:21:00.347555 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.341165    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9bm4\" (UniqueName: \"kubernetes.io/projected/2382715c-e2ff-4ef7-983f-0bebcbac420c-kube-api-access-x9bm4\") pod \"payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk\" (UID: \"2382715c-e2ff-4ef7-983f-0bebcbac420c\") " pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"
Sep 06 05:21:00.441774 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.441732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9bm4\" (UniqueName: \"kubernetes.io/projected/2382715c-e2ff-4ef7-983f-0bebcbac420c-kube-api-access-x9bm4\") pod \"payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk\" (UID: \"2382715c-e2ff-4ef7-983f-0bebcbac420c\") " pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"
Sep 06 05:21:00.441954 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.441787    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2382715c-e2ff-4ef7-983f-0bebcbac420c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk\" (UID: \"2382715c-e2ff-4ef7-983f-0bebcbac420c\") " pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"
Sep 06 05:21:00.442575 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.442548    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2382715c-e2ff-4ef7-983f-0bebcbac420c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk\" (UID: \"2382715c-e2ff-4ef7-983f-0bebcbac420c\") " pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"
Sep 06 05:21:00.449306 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.449283    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9bm4\" (UniqueName: \"kubernetes.io/projected/2382715c-e2ff-4ef7-983f-0bebcbac420c-kube-api-access-x9bm4\") pod \"payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk\" (UID: \"2382715c-e2ff-4ef7-983f-0bebcbac420c\") " pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"
Sep 06 05:21:00.546952 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.546917    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"
Sep 06 05:21:00.820164 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:00.820131    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"]
Sep 06 05:21:00.822005 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:21:00.821978    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2382715c_e2ff_4ef7_983f_0bebcbac420c.slice/crio-4d679d0f73fee74f60eef8fd5edfb965fbad20a821e68f5aaf99391bde9830cc WatchSource:0}: Error finding container 4d679d0f73fee74f60eef8fd5edfb965fbad20a821e68f5aaf99391bde9830cc: Status 404 returned error can't find the container with id 4d679d0f73fee74f60eef8fd5edfb965fbad20a821e68f5aaf99391bde9830cc
Sep 06 05:21:01.016426 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:01.016391    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk" event={"ID":"2382715c-e2ff-4ef7-983f-0bebcbac420c","Type":"ContainerStarted","Data":"a0df433eb2d2d00ab1db032f482c7b96a91c2dbeb966171b7ff20751152ed817"}
Sep 06 05:21:01.016590 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:01.016450    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk" event={"ID":"2382715c-e2ff-4ef7-983f-0bebcbac420c","Type":"ContainerStarted","Data":"4d679d0f73fee74f60eef8fd5edfb965fbad20a821e68f5aaf99391bde9830cc"}
Sep 06 05:21:01.016666 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:01.016602    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"
Sep 06 05:21:01.032956 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:01.032901    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk" podStartSLOduration=1.032887316 podStartE2EDuration="1.032887316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:21:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:21:01.030537269 +0000 UTC m=+1528.399992515" watchObservedRunningTime="2026-09-06 05:21:01.032887316 +0000 UTC m=+1528.402342562"
Sep 06 05:21:09.468435 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:09.468399    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:12.023856 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:12.023827    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"
Sep 06 05:21:23.458733 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:23.458696    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:23.929863 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:23.929828    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:24.253963 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:24.253886    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:24.329180 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:24.329149    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:24.879294 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:24.879263    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:27.622488 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:27.622438    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:27.715196 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:27.715146    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:28.842321 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:28.842286    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"]
Sep 06 05:21:28.846144 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:28.846117    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:28.848876 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:28.848358    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-596168-serving-cert\""
Sep 06 05:21:28.857143 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:28.857114    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"]
Sep 06 05:21:28.878457 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:28.878412    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:28.939767 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:28.939732    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-maas-api-tls\") pod \"maas-api-e2e-api-b-596168-646d8659dc-v5kz2\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:28.939971 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:28.939864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx95r\" (UniqueName: \"kubernetes.io/projected/4782f5d5-fa27-41a9-854c-81464aa2d38f-kube-api-access-gx95r\") pod \"maas-api-e2e-api-b-596168-646d8659dc-v5kz2\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:28.939971 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:28.939921    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4782f5d5-fa27-41a9-854c-81464aa2d38f-openshift-service-ca\") pod \"maas-api-e2e-api-b-596168-646d8659dc-v5kz2\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:28.940122 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:28.940004    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-metrics-certs\") pod \"maas-api-e2e-api-b-596168-646d8659dc-v5kz2\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:28.979280 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:28.979243    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:29.041008 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.040964    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx95r\" (UniqueName: \"kubernetes.io/projected/4782f5d5-fa27-41a9-854c-81464aa2d38f-kube-api-access-gx95r\") pod \"maas-api-e2e-api-b-596168-646d8659dc-v5kz2\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:29.041227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.041064    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4782f5d5-fa27-41a9-854c-81464aa2d38f-openshift-service-ca\") pod \"maas-api-e2e-api-b-596168-646d8659dc-v5kz2\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:29.041227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.041124    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-metrics-certs\") pod \"maas-api-e2e-api-b-596168-646d8659dc-v5kz2\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:29.041227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.041174    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-maas-api-tls\") pod \"maas-api-e2e-api-b-596168-646d8659dc-v5kz2\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:29.041362 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:21:29.041342    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-596168-serving-cert: secret "maas-api-e2e-api-b-596168-serving-cert" not found
Sep 06 05:21:29.041418 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:21:29.041407    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-maas-api-tls podName:4782f5d5-fa27-41a9-854c-81464aa2d38f nodeName:}" failed. No retries permitted until 2026-09-06 05:21:29.541389781 +0000 UTC m=+1556.910845004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-maas-api-tls") pod "maas-api-e2e-api-b-596168-646d8659dc-v5kz2" (UID: "4782f5d5-fa27-41a9-854c-81464aa2d38f") : secret "maas-api-e2e-api-b-596168-serving-cert" not found
Sep 06 05:21:29.041821 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.041799    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4782f5d5-fa27-41a9-854c-81464aa2d38f-openshift-service-ca\") pod \"maas-api-e2e-api-b-596168-646d8659dc-v5kz2\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:29.043557 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.043537    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-metrics-certs\") pod \"maas-api-e2e-api-b-596168-646d8659dc-v5kz2\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:29.048906 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.048876    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx95r\" (UniqueName: \"kubernetes.io/projected/4782f5d5-fa27-41a9-854c-81464aa2d38f-kube-api-access-gx95r\") pod \"maas-api-e2e-api-b-596168-646d8659dc-v5kz2\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:29.105411 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.105330    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:29.305423 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.305391    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:29.400100 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.400067    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:29.404096 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.404071    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"]
Sep 06 05:21:29.407967 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.407949    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"
Sep 06 05:21:29.410239 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.410217    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-596168\""
Sep 06 05:21:29.410337 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.410321    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-596168-dockercfg-ztz62\""
Sep 06 05:21:29.418120 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.418093    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"]
Sep 06 05:21:29.435668 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.435633    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"]
Sep 06 05:21:29.439287 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.439262    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"
Sep 06 05:21:29.445471 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.445443    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6225ded-788d-4a89-b327-c1a5d727213c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd\" (UID: \"c6225ded-788d-4a89-b327-c1a5d727213c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"
Sep 06 05:21:29.445627 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.445500    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bvnw\" (UniqueName: \"kubernetes.io/projected/c6225ded-788d-4a89-b327-c1a5d727213c-kube-api-access-2bvnw\") pod \"payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd\" (UID: \"c6225ded-788d-4a89-b327-c1a5d727213c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"
Sep 06 05:21:29.446594 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.446574    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"]
Sep 06 05:21:29.547092 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.547057    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-maas-api-tls\") pod \"maas-api-e2e-api-b-596168-646d8659dc-v5kz2\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:29.547288 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.547119    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhnbm\" (UniqueName: \"kubernetes.io/projected/1832b4be-8f46-425b-a040-e0b59f182a5f-kube-api-access-mhnbm\") pod \"payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c\" (UID: \"1832b4be-8f46-425b-a040-e0b59f182a5f\") " pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"
Sep 06 05:21:29.547288 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.547177    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6225ded-788d-4a89-b327-c1a5d727213c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd\" (UID: \"c6225ded-788d-4a89-b327-c1a5d727213c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"
Sep 06 05:21:29.547288 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.547219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bvnw\" (UniqueName: \"kubernetes.io/projected/c6225ded-788d-4a89-b327-c1a5d727213c-kube-api-access-2bvnw\") pod \"payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd\" (UID: \"c6225ded-788d-4a89-b327-c1a5d727213c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"
Sep 06 05:21:29.547471 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.547288    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1832b4be-8f46-425b-a040-e0b59f182a5f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c\" (UID: \"1832b4be-8f46-425b-a040-e0b59f182a5f\") " pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"
Sep 06 05:21:29.547899 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.547875    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6225ded-788d-4a89-b327-c1a5d727213c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd\" (UID: \"c6225ded-788d-4a89-b327-c1a5d727213c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"
Sep 06 05:21:29.549616 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.549589    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-maas-api-tls\") pod \"maas-api-e2e-api-b-596168-646d8659dc-v5kz2\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:29.554406 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.554383    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bvnw\" (UniqueName: \"kubernetes.io/projected/c6225ded-788d-4a89-b327-c1a5d727213c-kube-api-access-2bvnw\") pod \"payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd\" (UID: \"c6225ded-788d-4a89-b327-c1a5d727213c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"
Sep 06 05:21:29.648508 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.648470    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhnbm\" (UniqueName: \"kubernetes.io/projected/1832b4be-8f46-425b-a040-e0b59f182a5f-kube-api-access-mhnbm\") pod \"payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c\" (UID: \"1832b4be-8f46-425b-a040-e0b59f182a5f\") " pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"
Sep 06 05:21:29.648699 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.648540    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1832b4be-8f46-425b-a040-e0b59f182a5f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c\" (UID: \"1832b4be-8f46-425b-a040-e0b59f182a5f\") " pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"
Sep 06 05:21:29.649324 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.649299    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1832b4be-8f46-425b-a040-e0b59f182a5f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c\" (UID: \"1832b4be-8f46-425b-a040-e0b59f182a5f\") " pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"
Sep 06 05:21:29.656035 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.655967    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhnbm\" (UniqueName: \"kubernetes.io/projected/1832b4be-8f46-425b-a040-e0b59f182a5f-kube-api-access-mhnbm\") pod \"payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c\" (UID: \"1832b4be-8f46-425b-a040-e0b59f182a5f\") " pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"
Sep 06 05:21:29.720265 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.720230    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"
Sep 06 05:21:29.753180 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.753143    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"
Sep 06 05:21:29.760179 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.760147    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:29.996621 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:29.996594    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"]
Sep 06 05:21:29.998546 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:21:29.998518    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6225ded_788d_4a89_b327_c1a5d727213c.slice/crio-b70ef74b650b18be1dd20a4534f7be6aae1d861c5f30551da6dc0da0bbac0361 WatchSource:0}: Error finding container b70ef74b650b18be1dd20a4534f7be6aae1d861c5f30551da6dc0da0bbac0361: Status 404 returned error can't find the container with id b70ef74b650b18be1dd20a4534f7be6aae1d861c5f30551da6dc0da0bbac0361
Sep 06 05:21:30.041833 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:30.041803    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"]
Sep 06 05:21:30.043986 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:21:30.043958    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1832b4be_8f46_425b_a040_e0b59f182a5f.slice/crio-4e451f0974526d7b793b427b5cdcedb9b8f7bb03f2a274f16e4fa680da943efd WatchSource:0}: Error finding container 4e451f0974526d7b793b427b5cdcedb9b8f7bb03f2a274f16e4fa680da943efd: Status 404 returned error can't find the container with id 4e451f0974526d7b793b427b5cdcedb9b8f7bb03f2a274f16e4fa680da943efd
Sep 06 05:21:30.060151 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:30.060128    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"]
Sep 06 05:21:30.063291 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:21:30.063180    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4782f5d5_fa27_41a9_854c_81464aa2d38f.slice/crio-5de434709380ad884d60592cfb0806b16bdd8966baf2c951a9e0eac756295293 WatchSource:0}: Error finding container 5de434709380ad884d60592cfb0806b16bdd8966baf2c951a9e0eac756295293: Status 404 returned error can't find the container with id 5de434709380ad884d60592cfb0806b16bdd8966baf2c951a9e0eac756295293
Sep 06 05:21:30.064778 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:30.064756    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 05:21:30.182216 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:30.182169    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd" event={"ID":"c6225ded-788d-4a89-b327-c1a5d727213c","Type":"ContainerStarted","Data":"315243f7df7776ac9a92adeafbd9a6394c2db72c92269af5b3b14e8a6bb676a0"}
Sep 06 05:21:30.182409 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:30.182233    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"
Sep 06 05:21:30.182409 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:30.182258    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd" event={"ID":"c6225ded-788d-4a89-b327-c1a5d727213c","Type":"ContainerStarted","Data":"b70ef74b650b18be1dd20a4534f7be6aae1d861c5f30551da6dc0da0bbac0361"}
Sep 06 05:21:30.183909 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:30.183879    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c" event={"ID":"1832b4be-8f46-425b-a040-e0b59f182a5f","Type":"ContainerStarted","Data":"33c2eef6a366ae0bb57c1014cff96bf467f2cfefd372103125e71e7d7cbce8ad"}
Sep 06 05:21:30.184060 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:30.183935    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c" event={"ID":"1832b4be-8f46-425b-a040-e0b59f182a5f","Type":"ContainerStarted","Data":"4e451f0974526d7b793b427b5cdcedb9b8f7bb03f2a274f16e4fa680da943efd"}
Sep 06 05:21:30.184060 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:30.183968    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"
Sep 06 05:21:30.185324 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:30.185301    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2" event={"ID":"4782f5d5-fa27-41a9-854c-81464aa2d38f","Type":"ContainerStarted","Data":"5de434709380ad884d60592cfb0806b16bdd8966baf2c951a9e0eac756295293"}
Sep 06 05:21:30.202743 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:30.201536    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd" podStartSLOduration=1.201516944 podStartE2EDuration="1.201516944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:21:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:21:30.201330514 +0000 UTC m=+1557.570785762" watchObservedRunningTime="2026-09-06 05:21:30.201516944 +0000 UTC m=+1557.570972191"
Sep 06 05:21:30.216842 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:30.216794    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c" podStartSLOduration=1.216780487 podStartE2EDuration="1.216780487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:21:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:21:30.215003943 +0000 UTC m=+1557.584459188" watchObservedRunningTime="2026-09-06 05:21:30.216780487 +0000 UTC m=+1557.586235733"
Sep 06 05:21:31.193447 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:31.193401    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2" event={"ID":"4782f5d5-fa27-41a9-854c-81464aa2d38f","Type":"ContainerStarted","Data":"e5cd4002f05b84f46ba1dacd18725072f989e44b3290da22f9cb109798922f32"}
Sep 06 05:21:31.210365 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:31.210311    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2" podStartSLOduration=2.879115591 podStartE2EDuration="3.210292744s" totalImagesPullingTime="331.177153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:21:31.208810427 +0000 UTC m=+1558.578265674" watchObservedRunningTime="2026-09-06 05:21:31.210292744 +0000 UTC m=+1558.579747991"
Sep 06 05:21:32.199183 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:32.199151    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:33.387538 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:33.387508    2578 scope.go:117] "RemoveContainer" containerID="18214127544cb101040539af8f29e58db527e61e89a5f530c9e718a371b51f75"
Sep 06 05:21:38.209717 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:38.209687    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:21:41.195778 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:41.195747    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"
Sep 06 05:21:41.196267 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:41.195821    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"
Sep 06 05:21:52.591842 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:52.591802    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:53.565455 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:53.565413    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:53.776524 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:53.776493    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:54.570035 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:54.569981    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:55.273041 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:55.272991    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:55.970695 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:55.970661    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:21:57.974265 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:21:57.974236    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:10.947095 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:10.947062    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:10.971524 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:10.971489    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"]
Sep 06 05:22:10.971818 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:10.971771    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk" podUID="2382715c-e2ff-4ef7-983f-0bebcbac420c" containerName="payload-processing" containerID="cri-o://a0df433eb2d2d00ab1db032f482c7b96a91c2dbeb966171b7ff20751152ed817" gracePeriod=30
Sep 06 05:22:11.047301 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.047274    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:11.117198 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.117165    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:11.171491 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.171457    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:11.228313 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.228283    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"
Sep 06 05:22:11.262100 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.262069    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9bm4\" (UniqueName: \"kubernetes.io/projected/2382715c-e2ff-4ef7-983f-0bebcbac420c-kube-api-access-x9bm4\") pod \"2382715c-e2ff-4ef7-983f-0bebcbac420c\" (UID: \"2382715c-e2ff-4ef7-983f-0bebcbac420c\") "
Sep 06 05:22:11.262100 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.262104    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2382715c-e2ff-4ef7-983f-0bebcbac420c-plugins-config-volume\") pod \"2382715c-e2ff-4ef7-983f-0bebcbac420c\" (UID: \"2382715c-e2ff-4ef7-983f-0bebcbac420c\") "
Sep 06 05:22:11.262614 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.262568    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2382715c-e2ff-4ef7-983f-0bebcbac420c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2382715c-e2ff-4ef7-983f-0bebcbac420c" (UID: "2382715c-e2ff-4ef7-983f-0bebcbac420c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:22:11.264271 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.264245    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2382715c-e2ff-4ef7-983f-0bebcbac420c-kube-api-access-x9bm4" (OuterVolumeSpecName: "kube-api-access-x9bm4") pod "2382715c-e2ff-4ef7-983f-0bebcbac420c" (UID: "2382715c-e2ff-4ef7-983f-0bebcbac420c"). InnerVolumeSpecName "kube-api-access-x9bm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:22:11.363272 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.363236    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9bm4\" (UniqueName: \"kubernetes.io/projected/2382715c-e2ff-4ef7-983f-0bebcbac420c-kube-api-access-x9bm4\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:22:11.363272 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.363265    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2382715c-e2ff-4ef7-983f-0bebcbac420c-plugins-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:22:11.422446 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.422410    2578 generic.go:356] "Generic (PLEG): container finished" podID="2382715c-e2ff-4ef7-983f-0bebcbac420c" containerID="a0df433eb2d2d00ab1db032f482c7b96a91c2dbeb966171b7ff20751152ed817" exitCode=0
Sep 06 05:22:11.422598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.422467    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"
Sep 06 05:22:11.422598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.422493    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk" event={"ID":"2382715c-e2ff-4ef7-983f-0bebcbac420c","Type":"ContainerDied","Data":"a0df433eb2d2d00ab1db032f482c7b96a91c2dbeb966171b7ff20751152ed817"}
Sep 06 05:22:11.422598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.422549    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk" event={"ID":"2382715c-e2ff-4ef7-983f-0bebcbac420c","Type":"ContainerDied","Data":"4d679d0f73fee74f60eef8fd5edfb965fbad20a821e68f5aaf99391bde9830cc"}
Sep 06 05:22:11.422598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.422578    2578 scope.go:117] "RemoveContainer" containerID="a0df433eb2d2d00ab1db032f482c7b96a91c2dbeb966171b7ff20751152ed817"
Sep 06 05:22:11.439672 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.439651    2578 scope.go:117] "RemoveContainer" containerID="a0df433eb2d2d00ab1db032f482c7b96a91c2dbeb966171b7ff20751152ed817"
Sep 06 05:22:11.439961 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:22:11.439940    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0df433eb2d2d00ab1db032f482c7b96a91c2dbeb966171b7ff20751152ed817\": container with ID starting with a0df433eb2d2d00ab1db032f482c7b96a91c2dbeb966171b7ff20751152ed817 not found: ID does not exist" containerID="a0df433eb2d2d00ab1db032f482c7b96a91c2dbeb966171b7ff20751152ed817"
Sep 06 05:22:11.440067 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.439969    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0df433eb2d2d00ab1db032f482c7b96a91c2dbeb966171b7ff20751152ed817"} err="failed to get container status \"a0df433eb2d2d00ab1db032f482c7b96a91c2dbeb966171b7ff20751152ed817\": rpc error: code = NotFound desc = could not find container \"a0df433eb2d2d00ab1db032f482c7b96a91c2dbeb966171b7ff20751152ed817\": container with ID starting with a0df433eb2d2d00ab1db032f482c7b96a91c2dbeb966171b7ff20751152ed817 not found: ID does not exist"
Sep 06 05:22:11.448093 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.448069    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"]
Sep 06 05:22:11.450964 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.450944    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-71403e-79f47797d7-dmhrk"]
Sep 06 05:22:11.488707 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:11.488646    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:13.154315 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:13.154283    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2382715c-e2ff-4ef7-983f-0bebcbac420c" path="/var/lib/kubelet/pods/2382715c-e2ff-4ef7-983f-0bebcbac420c/volumes"
Sep 06 05:22:17.090649 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:17.090603    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:17.190867 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:17.190834    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:17.350756 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:17.350678    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:17.965805 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:17.965761    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:19.660289 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:19.660253    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:23.854110 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:23.854077    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:27.858494 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:27.858408    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:33.353934 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.353900    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"]
Sep 06 05:22:33.354432 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.354192    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2" podUID="4782f5d5-fa27-41a9-854c-81464aa2d38f" containerName="maas-api" containerID="cri-o://e5cd4002f05b84f46ba1dacd18725072f989e44b3290da22f9cb109798922f32" gracePeriod=30
Sep 06 05:22:33.381426 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.381390    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:33.409614 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.409554    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"]
Sep 06 05:22:33.410122 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.410090    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c" podUID="1832b4be-8f46-425b-a040-e0b59f182a5f" containerName="payload-processing" containerID="cri-o://33c2eef6a366ae0bb57c1014cff96bf467f2cfefd372103125e71e7d7cbce8ad" gracePeriod=30
Sep 06 05:22:33.412595 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.412564    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"]
Sep 06 05:22:33.412890 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.412834    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd" podUID="c6225ded-788d-4a89-b327-c1a5d727213c" containerName="payload-pre-processing" containerID="cri-o://315243f7df7776ac9a92adeafbd9a6394c2db72c92269af5b3b14e8a6bb676a0" gracePeriod=30
Sep 06 05:22:33.448275 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.448243    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:33.505389 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.505361    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:33.557474 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.557437    2578 generic.go:356] "Generic (PLEG): container finished" podID="c6225ded-788d-4a89-b327-c1a5d727213c" containerID="315243f7df7776ac9a92adeafbd9a6394c2db72c92269af5b3b14e8a6bb676a0" exitCode=0
Sep 06 05:22:33.557631 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.557545    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd" event={"ID":"c6225ded-788d-4a89-b327-c1a5d727213c","Type":"ContainerDied","Data":"315243f7df7776ac9a92adeafbd9a6394c2db72c92269af5b3b14e8a6bb676a0"}
Sep 06 05:22:33.560683 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.560617    2578 generic.go:356] "Generic (PLEG): container finished" podID="1832b4be-8f46-425b-a040-e0b59f182a5f" containerID="33c2eef6a366ae0bb57c1014cff96bf467f2cfefd372103125e71e7d7cbce8ad" exitCode=0
Sep 06 05:22:33.560840 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.560715    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c" event={"ID":"1832b4be-8f46-425b-a040-e0b59f182a5f","Type":"ContainerDied","Data":"33c2eef6a366ae0bb57c1014cff96bf467f2cfefd372103125e71e7d7cbce8ad"}
Sep 06 05:22:33.563127 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.563079    2578 generic.go:356] "Generic (PLEG): container finished" podID="4782f5d5-fa27-41a9-854c-81464aa2d38f" containerID="e5cd4002f05b84f46ba1dacd18725072f989e44b3290da22f9cb109798922f32" exitCode=0
Sep 06 05:22:33.563127 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.563117    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2" event={"ID":"4782f5d5-fa27-41a9-854c-81464aa2d38f","Type":"ContainerDied","Data":"e5cd4002f05b84f46ba1dacd18725072f989e44b3290da22f9cb109798922f32"}
Sep 06 05:22:33.563613 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.563586    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:33.693644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.693604    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:22:33.697468 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.697440    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4782f5d5-fa27-41a9-854c-81464aa2d38f-openshift-service-ca\") pod \"4782f5d5-fa27-41a9-854c-81464aa2d38f\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") "
Sep 06 05:22:33.697624 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.697521    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-metrics-certs\") pod \"4782f5d5-fa27-41a9-854c-81464aa2d38f\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") "
Sep 06 05:22:33.697624 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.697583    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-maas-api-tls\") pod \"4782f5d5-fa27-41a9-854c-81464aa2d38f\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") "
Sep 06 05:22:33.697747 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.697657    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx95r\" (UniqueName: \"kubernetes.io/projected/4782f5d5-fa27-41a9-854c-81464aa2d38f-kube-api-access-gx95r\") pod \"4782f5d5-fa27-41a9-854c-81464aa2d38f\" (UID: \"4782f5d5-fa27-41a9-854c-81464aa2d38f\") "
Sep 06 05:22:33.698273 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.698243    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4782f5d5-fa27-41a9-854c-81464aa2d38f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4782f5d5-fa27-41a9-854c-81464aa2d38f" (UID: "4782f5d5-fa27-41a9-854c-81464aa2d38f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:22:33.701387 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.701280    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4782f5d5-fa27-41a9-854c-81464aa2d38f" (UID: "4782f5d5-fa27-41a9-854c-81464aa2d38f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:22:33.701387 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.701357    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4782f5d5-fa27-41a9-854c-81464aa2d38f" (UID: "4782f5d5-fa27-41a9-854c-81464aa2d38f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:22:33.701496 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.701444    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4782f5d5-fa27-41a9-854c-81464aa2d38f-kube-api-access-gx95r" (OuterVolumeSpecName: "kube-api-access-gx95r") pod "4782f5d5-fa27-41a9-854c-81464aa2d38f" (UID: "4782f5d5-fa27-41a9-854c-81464aa2d38f"). InnerVolumeSpecName "kube-api-access-gx95r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:22:33.723930 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.723901    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"
Sep 06 05:22:33.734771 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.734746    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"
Sep 06 05:22:33.798416 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.798382    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhnbm\" (UniqueName: \"kubernetes.io/projected/1832b4be-8f46-425b-a040-e0b59f182a5f-kube-api-access-mhnbm\") pod \"1832b4be-8f46-425b-a040-e0b59f182a5f\" (UID: \"1832b4be-8f46-425b-a040-e0b59f182a5f\") "
Sep 06 05:22:33.798578 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.798447    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6225ded-788d-4a89-b327-c1a5d727213c-plugins-config-volume\") pod \"c6225ded-788d-4a89-b327-c1a5d727213c\" (UID: \"c6225ded-788d-4a89-b327-c1a5d727213c\") "
Sep 06 05:22:33.798578 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.798487    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1832b4be-8f46-425b-a040-e0b59f182a5f-plugins-config-volume\") pod \"1832b4be-8f46-425b-a040-e0b59f182a5f\" (UID: \"1832b4be-8f46-425b-a040-e0b59f182a5f\") "
Sep 06 05:22:33.798578 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.798541    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bvnw\" (UniqueName: \"kubernetes.io/projected/c6225ded-788d-4a89-b327-c1a5d727213c-kube-api-access-2bvnw\") pod \"c6225ded-788d-4a89-b327-c1a5d727213c\" (UID: \"c6225ded-788d-4a89-b327-c1a5d727213c\") "
Sep 06 05:22:33.798863 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.798843    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4782f5d5-fa27-41a9-854c-81464aa2d38f-openshift-service-ca\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:22:33.798910 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.798869    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-metrics-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:22:33.798910 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.798883    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4782f5d5-fa27-41a9-854c-81464aa2d38f-maas-api-tls\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:22:33.798910 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.798899    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx95r\" (UniqueName: \"kubernetes.io/projected/4782f5d5-fa27-41a9-854c-81464aa2d38f-kube-api-access-gx95r\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:22:33.798910 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.798860    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1832b4be-8f46-425b-a040-e0b59f182a5f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1832b4be-8f46-425b-a040-e0b59f182a5f" (UID: "1832b4be-8f46-425b-a040-e0b59f182a5f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:22:33.799066 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.798878    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6225ded-788d-4a89-b327-c1a5d727213c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c6225ded-788d-4a89-b327-c1a5d727213c" (UID: "c6225ded-788d-4a89-b327-c1a5d727213c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:22:33.800681 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.800657    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1832b4be-8f46-425b-a040-e0b59f182a5f-kube-api-access-mhnbm" (OuterVolumeSpecName: "kube-api-access-mhnbm") pod "1832b4be-8f46-425b-a040-e0b59f182a5f" (UID: "1832b4be-8f46-425b-a040-e0b59f182a5f"). InnerVolumeSpecName "kube-api-access-mhnbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:22:33.800681 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.800670    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6225ded-788d-4a89-b327-c1a5d727213c-kube-api-access-2bvnw" (OuterVolumeSpecName: "kube-api-access-2bvnw") pod "c6225ded-788d-4a89-b327-c1a5d727213c" (UID: "c6225ded-788d-4a89-b327-c1a5d727213c"). InnerVolumeSpecName "kube-api-access-2bvnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:22:33.899957 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.899870    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bvnw\" (UniqueName: \"kubernetes.io/projected/c6225ded-788d-4a89-b327-c1a5d727213c-kube-api-access-2bvnw\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:22:33.899957 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.899903    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhnbm\" (UniqueName: \"kubernetes.io/projected/1832b4be-8f46-425b-a040-e0b59f182a5f-kube-api-access-mhnbm\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:22:33.899957 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.899925    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6225ded-788d-4a89-b327-c1a5d727213c-plugins-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:22:33.899957 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.899934    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1832b4be-8f46-425b-a040-e0b59f182a5f-plugins-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:22:33.905244 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:33.905215    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:34.570042 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.570003    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"
Sep 06 05:22:34.570042 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.570012    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd" event={"ID":"c6225ded-788d-4a89-b327-c1a5d727213c","Type":"ContainerDied","Data":"b70ef74b650b18be1dd20a4534f7be6aae1d861c5f30551da6dc0da0bbac0361"}
Sep 06 05:22:34.570554 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.570088    2578 scope.go:117] "RemoveContainer" containerID="315243f7df7776ac9a92adeafbd9a6394c2db72c92269af5b3b14e8a6bb676a0"
Sep 06 05:22:34.571564 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.571540    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c" event={"ID":"1832b4be-8f46-425b-a040-e0b59f182a5f","Type":"ContainerDied","Data":"4e451f0974526d7b793b427b5cdcedb9b8f7bb03f2a274f16e4fa680da943efd"}
Sep 06 05:22:34.571687 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.571546    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"
Sep 06 05:22:34.573132 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.573105    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"
Sep 06 05:22:34.573243 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.573109    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2" event={"ID":"4782f5d5-fa27-41a9-854c-81464aa2d38f","Type":"ContainerDied","Data":"5de434709380ad884d60592cfb0806b16bdd8966baf2c951a9e0eac756295293"}
Sep 06 05:22:34.579422 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.579404    2578 scope.go:117] "RemoveContainer" containerID="33c2eef6a366ae0bb57c1014cff96bf467f2cfefd372103125e71e7d7cbce8ad"
Sep 06 05:22:34.589023 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.588993    2578 scope.go:117] "RemoveContainer" containerID="e5cd4002f05b84f46ba1dacd18725072f989e44b3290da22f9cb109798922f32"
Sep 06 05:22:34.611116 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.611093    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"]
Sep 06 05:22:34.619190 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.619166    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-596168-646d8659dc-v5kz2"]
Sep 06 05:22:34.638751 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.638699    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"]
Sep 06 05:22:34.654128 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.654105    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-596168-b5c6dd69f-xjl4c"]
Sep 06 05:22:34.672352 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.672322    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"]
Sep 06 05:22:34.678107 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:34.678085    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-596168-555f7bb5bb-5fwqd"]
Sep 06 05:22:35.157248 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:35.157214    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1832b4be-8f46-425b-a040-e0b59f182a5f" path="/var/lib/kubelet/pods/1832b4be-8f46-425b-a040-e0b59f182a5f/volumes"
Sep 06 05:22:35.157825 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:35.157806    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4782f5d5-fa27-41a9-854c-81464aa2d38f" path="/var/lib/kubelet/pods/4782f5d5-fa27-41a9-854c-81464aa2d38f/volumes"
Sep 06 05:22:35.158340 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:35.158321    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6225ded-788d-4a89-b327-c1a5d727213c" path="/var/lib/kubelet/pods/c6225ded-788d-4a89-b327-c1a5d727213c/volumes"
Sep 06 05:22:39.527252 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:39.527222    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:39.627422 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:39.627373    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:39.785476 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:39.785368    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:40.369114 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:40.369061    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:50.557307 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:50.557269    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:50.620736 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:50.620697    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:50.680291 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:50.680260    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:50.735687 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:50.735648    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:51.081619 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:51.081583    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:55.875324 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:55.875295    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:56.806900 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:56.806862    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:56.907179 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:56.907139    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:57.043653 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:57.043622    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"]
Sep 06 05:22:57.043897 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:57.043874    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx" podUID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerName="istio-proxy" containerID="cri-o://752455ebc506e2e5791ff0ebb0ed5da8086f298f6f3b5a74edb254fd8391314f" gracePeriod=30
Sep 06 05:22:57.064389 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:57.064322    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:22:57.576622 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:22:57.576580    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:02.311407 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.311349    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:23:02.473906 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.473865    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-workload-certs\") pod \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") "
Sep 06 05:23:02.474113 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.473935    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istiod-ca-cert\") pod \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") "
Sep 06 05:23:02.474113 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.473953    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9dsj\" (UniqueName: \"kubernetes.io/projected/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-kube-api-access-d9dsj\") pod \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") "
Sep 06 05:23:02.474113 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.473972    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-envoy\") pod \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") "
Sep 06 05:23:02.474113 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474011    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-credential-socket\") pod \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") "
Sep 06 05:23:02.474331 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474130    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-data\") pod \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") "
Sep 06 05:23:02.474331 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474170    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-workload-socket\") pod \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") "
Sep 06 05:23:02.474331 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474202    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" (UID: "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:23:02.474331 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474254    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-podinfo\") pod \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") "
Sep 06 05:23:02.474331 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474320    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-token\") pod \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\" (UID: \"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9\") "
Sep 06 05:23:02.474331 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474322    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" (UID: "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:23:02.474647 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474367    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-data" (OuterVolumeSpecName: "istio-data") pod "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" (UID: "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:23:02.474647 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474489    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" (UID: "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:23:02.474647 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474512    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" (UID: "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:23:02.474811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474711    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istiod-ca-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:02.474811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474729    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-credential-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:02.474811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474743    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-data\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:02.474811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474754    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-workload-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:02.474811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.474762    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-workload-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:02.476317 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.476293    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-token" (OuterVolumeSpecName: "istio-token") pod "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" (UID: "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:23:02.476416 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.476346    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" (UID: "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:23:02.476633 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.476616    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" (UID: "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:23:02.476686 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.476628    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-kube-api-access-d9dsj" (OuterVolumeSpecName: "kube-api-access-d9dsj") pod "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" (UID: "ee44c28c-f2b1-4e58-a413-6fa792dcf4e9"). InnerVolumeSpecName "kube-api-access-d9dsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:23:02.576115 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.576011    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9dsj\" (UniqueName: \"kubernetes.io/projected/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-kube-api-access-d9dsj\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:02.576115 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.576059    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-envoy\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:02.576115 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.576074    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-podinfo\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:02.576115 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.576085    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9-istio-token\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:02.738009 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.737975    2578 generic.go:356] "Generic (PLEG): container finished" podID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerID="752455ebc506e2e5791ff0ebb0ed5da8086f298f6f3b5a74edb254fd8391314f" exitCode=0
Sep 06 05:23:02.738198 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.738038    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"
Sep 06 05:23:02.738198 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.738038    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx" event={"ID":"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9","Type":"ContainerDied","Data":"752455ebc506e2e5791ff0ebb0ed5da8086f298f6f3b5a74edb254fd8391314f"}
Sep 06 05:23:02.738198 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.738151    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx" event={"ID":"ee44c28c-f2b1-4e58-a413-6fa792dcf4e9","Type":"ContainerDied","Data":"1454b99bda9a25fdd3c71dd175473ebadbb5d875080238f3cadb7ffc685ee0f7"}
Sep 06 05:23:02.738198 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.738175    2578 scope.go:117] "RemoveContainer" containerID="752455ebc506e2e5791ff0ebb0ed5da8086f298f6f3b5a74edb254fd8391314f"
Sep 06 05:23:02.747912 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.747892    2578 scope.go:117] "RemoveContainer" containerID="a744440c57adb1f90665578a79fd97e24b101678417cdbaf1fa5baa1d02e9843"
Sep 06 05:23:02.755917 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.755897    2578 scope.go:117] "RemoveContainer" containerID="752455ebc506e2e5791ff0ebb0ed5da8086f298f6f3b5a74edb254fd8391314f"
Sep 06 05:23:02.756232 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:23:02.756208    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"752455ebc506e2e5791ff0ebb0ed5da8086f298f6f3b5a74edb254fd8391314f\": container with ID starting with 752455ebc506e2e5791ff0ebb0ed5da8086f298f6f3b5a74edb254fd8391314f not found: ID does not exist" containerID="752455ebc506e2e5791ff0ebb0ed5da8086f298f6f3b5a74edb254fd8391314f"
Sep 06 05:23:02.756232 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.756237    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"752455ebc506e2e5791ff0ebb0ed5da8086f298f6f3b5a74edb254fd8391314f"} err="failed to get container status \"752455ebc506e2e5791ff0ebb0ed5da8086f298f6f3b5a74edb254fd8391314f\": rpc error: code = NotFound desc = could not find container \"752455ebc506e2e5791ff0ebb0ed5da8086f298f6f3b5a74edb254fd8391314f\": container with ID starting with 752455ebc506e2e5791ff0ebb0ed5da8086f298f6f3b5a74edb254fd8391314f not found: ID does not exist"
Sep 06 05:23:02.756516 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.756255    2578 scope.go:117] "RemoveContainer" containerID="a744440c57adb1f90665578a79fd97e24b101678417cdbaf1fa5baa1d02e9843"
Sep 06 05:23:02.756590 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:23:02.756560    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a744440c57adb1f90665578a79fd97e24b101678417cdbaf1fa5baa1d02e9843\": container with ID starting with a744440c57adb1f90665578a79fd97e24b101678417cdbaf1fa5baa1d02e9843 not found: ID does not exist" containerID="a744440c57adb1f90665578a79fd97e24b101678417cdbaf1fa5baa1d02e9843"
Sep 06 05:23:02.756647 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.756596    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a744440c57adb1f90665578a79fd97e24b101678417cdbaf1fa5baa1d02e9843"} err="failed to get container status \"a744440c57adb1f90665578a79fd97e24b101678417cdbaf1fa5baa1d02e9843\": rpc error: code = NotFound desc = could not find container \"a744440c57adb1f90665578a79fd97e24b101678417cdbaf1fa5baa1d02e9843\": container with ID starting with a744440c57adb1f90665578a79fd97e24b101678417cdbaf1fa5baa1d02e9843 not found: ID does not exist"
Sep 06 05:23:02.759276 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.759254    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"]
Sep 06 05:23:02.762674 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:02.762652    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e24961-openshift-default-5bbb8d648-wx7cx"]
Sep 06 05:23:03.154345 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:03.154315    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" path="/var/lib/kubelet/pods/ee44c28c-f2b1-4e58-a413-6fa792dcf4e9/volumes"
Sep 06 05:23:06.806100 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:06.806049    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:07.967385 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:07.967343    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"]
Sep 06 05:23:07.968148 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:07.967569    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58" podUID="43725b3f-a6f4-4f8d-b614-679e019d3cb8" containerName="maas-api" containerID="cri-o://728c1779e7bd4b10d9d2ce95b824e7b90444d29c57338b976f238849346e79fe" gracePeriod=30
Sep 06 05:23:07.988477 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:07.988440    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:08.008247 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.008214    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"]
Sep 06 05:23:08.008521 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.008472    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x" podUID="b3b9b742-ef50-4aac-90a4-01ebbc38590e" containerName="payload-processing" containerID="cri-o://d0d7340f6b5a4b741233aa46dd391820f37a35b9a6517d26e76326e374dd96ad" gracePeriod=30
Sep 06 05:23:08.086212 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.086181    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:08.152714 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.152678    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:08.204666 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.204533    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:08.275503 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.275480    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:23:08.278579 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.278559    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"
Sep 06 05:23:08.429158 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.429122    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-672wm\" (UniqueName: \"kubernetes.io/projected/b3b9b742-ef50-4aac-90a4-01ebbc38590e-kube-api-access-672wm\") pod \"b3b9b742-ef50-4aac-90a4-01ebbc38590e\" (UID: \"b3b9b742-ef50-4aac-90a4-01ebbc38590e\") "
Sep 06 05:23:08.429319 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.429173    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3b9b742-ef50-4aac-90a4-01ebbc38590e-plugins-config-volume\") pod \"b3b9b742-ef50-4aac-90a4-01ebbc38590e\" (UID: \"b3b9b742-ef50-4aac-90a4-01ebbc38590e\") "
Sep 06 05:23:08.429319 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.429213    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wrwd\" (UniqueName: \"kubernetes.io/projected/43725b3f-a6f4-4f8d-b614-679e019d3cb8-kube-api-access-8wrwd\") pod \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") "
Sep 06 05:23:08.429319 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.429252    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-maas-api-tls\") pod \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") "
Sep 06 05:23:08.429319 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.429313    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-metrics-certs\") pod \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") "
Sep 06 05:23:08.429525 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.429367    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43725b3f-a6f4-4f8d-b614-679e019d3cb8-openshift-service-ca\") pod \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\" (UID: \"43725b3f-a6f4-4f8d-b614-679e019d3cb8\") "
Sep 06 05:23:08.429627 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.429598    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3b9b742-ef50-4aac-90a4-01ebbc38590e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b3b9b742-ef50-4aac-90a4-01ebbc38590e" (UID: "b3b9b742-ef50-4aac-90a4-01ebbc38590e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:23:08.429710 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.429702    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3b9b742-ef50-4aac-90a4-01ebbc38590e-plugins-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:08.429815 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.429783    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43725b3f-a6f4-4f8d-b614-679e019d3cb8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "43725b3f-a6f4-4f8d-b614-679e019d3cb8" (UID: "43725b3f-a6f4-4f8d-b614-679e019d3cb8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:23:08.431531 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.431503    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3b9b742-ef50-4aac-90a4-01ebbc38590e-kube-api-access-672wm" (OuterVolumeSpecName: "kube-api-access-672wm") pod "b3b9b742-ef50-4aac-90a4-01ebbc38590e" (UID: "b3b9b742-ef50-4aac-90a4-01ebbc38590e"). InnerVolumeSpecName "kube-api-access-672wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:23:08.431653 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.431560    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "43725b3f-a6f4-4f8d-b614-679e019d3cb8" (UID: "43725b3f-a6f4-4f8d-b614-679e019d3cb8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:23:08.431653 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.431593    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "43725b3f-a6f4-4f8d-b614-679e019d3cb8" (UID: "43725b3f-a6f4-4f8d-b614-679e019d3cb8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:23:08.431653 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.431630    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43725b3f-a6f4-4f8d-b614-679e019d3cb8-kube-api-access-8wrwd" (OuterVolumeSpecName: "kube-api-access-8wrwd") pod "43725b3f-a6f4-4f8d-b614-679e019d3cb8" (UID: "43725b3f-a6f4-4f8d-b614-679e019d3cb8"). InnerVolumeSpecName "kube-api-access-8wrwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:23:08.487524 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.487499    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:08.530977 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.530894    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43725b3f-a6f4-4f8d-b614-679e019d3cb8-openshift-service-ca\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:08.530977 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.530924    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-672wm\" (UniqueName: \"kubernetes.io/projected/b3b9b742-ef50-4aac-90a4-01ebbc38590e-kube-api-access-672wm\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:08.530977 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.530934    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wrwd\" (UniqueName: \"kubernetes.io/projected/43725b3f-a6f4-4f8d-b614-679e019d3cb8-kube-api-access-8wrwd\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:08.530977 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.530945    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-maas-api-tls\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:08.530977 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.530955    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43725b3f-a6f4-4f8d-b614-679e019d3cb8-metrics-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:23:08.779148 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.779112    2578 generic.go:356] "Generic (PLEG): container finished" podID="43725b3f-a6f4-4f8d-b614-679e019d3cb8" containerID="728c1779e7bd4b10d9d2ce95b824e7b90444d29c57338b976f238849346e79fe" exitCode=0
Sep 06 05:23:08.779312 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.779156    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"
Sep 06 05:23:08.779312 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.779197    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58" event={"ID":"43725b3f-a6f4-4f8d-b614-679e019d3cb8","Type":"ContainerDied","Data":"728c1779e7bd4b10d9d2ce95b824e7b90444d29c57338b976f238849346e79fe"}
Sep 06 05:23:08.779312 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.779253    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58" event={"ID":"43725b3f-a6f4-4f8d-b614-679e019d3cb8","Type":"ContainerDied","Data":"10e4a905d28e912b3e3127811cc61c299266a70e165dd0303631aad075b9c6c6"}
Sep 06 05:23:08.779312 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.779280    2578 scope.go:117] "RemoveContainer" containerID="728c1779e7bd4b10d9d2ce95b824e7b90444d29c57338b976f238849346e79fe"
Sep 06 05:23:08.780584 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.780563    2578 generic.go:356] "Generic (PLEG): container finished" podID="b3b9b742-ef50-4aac-90a4-01ebbc38590e" containerID="d0d7340f6b5a4b741233aa46dd391820f37a35b9a6517d26e76326e374dd96ad" exitCode=0
Sep 06 05:23:08.780691 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.780587    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x" event={"ID":"b3b9b742-ef50-4aac-90a4-01ebbc38590e","Type":"ContainerDied","Data":"d0d7340f6b5a4b741233aa46dd391820f37a35b9a6517d26e76326e374dd96ad"}
Sep 06 05:23:08.780691 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.780629    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x" event={"ID":"b3b9b742-ef50-4aac-90a4-01ebbc38590e","Type":"ContainerDied","Data":"e7ee0a10fedabc9eb70730cc3af45d9c63369e8c328afc062a95076488b5f315"}
Sep 06 05:23:08.780691 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.780636    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"
Sep 06 05:23:08.788241 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.788223    2578 scope.go:117] "RemoveContainer" containerID="728c1779e7bd4b10d9d2ce95b824e7b90444d29c57338b976f238849346e79fe"
Sep 06 05:23:08.788732 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:23:08.788709    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"728c1779e7bd4b10d9d2ce95b824e7b90444d29c57338b976f238849346e79fe\": container with ID starting with 728c1779e7bd4b10d9d2ce95b824e7b90444d29c57338b976f238849346e79fe not found: ID does not exist" containerID="728c1779e7bd4b10d9d2ce95b824e7b90444d29c57338b976f238849346e79fe"
Sep 06 05:23:08.788868 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.788845    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"728c1779e7bd4b10d9d2ce95b824e7b90444d29c57338b976f238849346e79fe"} err="failed to get container status \"728c1779e7bd4b10d9d2ce95b824e7b90444d29c57338b976f238849346e79fe\": rpc error: code = NotFound desc = could not find container \"728c1779e7bd4b10d9d2ce95b824e7b90444d29c57338b976f238849346e79fe\": container with ID starting with 728c1779e7bd4b10d9d2ce95b824e7b90444d29c57338b976f238849346e79fe not found: ID does not exist"
Sep 06 05:23:08.788948 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.788918    2578 scope.go:117] "RemoveContainer" containerID="d0d7340f6b5a4b741233aa46dd391820f37a35b9a6517d26e76326e374dd96ad"
Sep 06 05:23:08.809268 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.809245    2578 scope.go:117] "RemoveContainer" containerID="d0d7340f6b5a4b741233aa46dd391820f37a35b9a6517d26e76326e374dd96ad"
Sep 06 05:23:08.809601 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:23:08.809581    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0d7340f6b5a4b741233aa46dd391820f37a35b9a6517d26e76326e374dd96ad\": container with ID starting with d0d7340f6b5a4b741233aa46dd391820f37a35b9a6517d26e76326e374dd96ad not found: ID does not exist" containerID="d0d7340f6b5a4b741233aa46dd391820f37a35b9a6517d26e76326e374dd96ad"
Sep 06 05:23:08.809601 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.809593    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"]
Sep 06 05:23:08.809702 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.809612    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0d7340f6b5a4b741233aa46dd391820f37a35b9a6517d26e76326e374dd96ad"} err="failed to get container status \"d0d7340f6b5a4b741233aa46dd391820f37a35b9a6517d26e76326e374dd96ad\": rpc error: code = NotFound desc = could not find container \"d0d7340f6b5a4b741233aa46dd391820f37a35b9a6517d26e76326e374dd96ad\": container with ID starting with d0d7340f6b5a4b741233aa46dd391820f37a35b9a6517d26e76326e374dd96ad not found: ID does not exist"
Sep 06 05:23:08.813028 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.812988    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-401f17-c779d4c6f-tn98x"]
Sep 06 05:23:08.821468 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.821448    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"]
Sep 06 05:23:08.824995 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:08.824963    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-401f17-f7bdfbd6-c7v58"]
Sep 06 05:23:09.154185 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:09.154153    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43725b3f-a6f4-4f8d-b614-679e019d3cb8" path="/var/lib/kubelet/pods/43725b3f-a6f4-4f8d-b614-679e019d3cb8/volumes"
Sep 06 05:23:09.154554 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:09.154537    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3b9b742-ef50-4aac-90a4-01ebbc38590e" path="/var/lib/kubelet/pods/b3b9b742-ef50-4aac-90a4-01ebbc38590e/volumes"
Sep 06 05:23:14.076769 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:14.076741    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:14.131662 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:14.131631    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:14.326373 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:14.326326    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:21.095617 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:21.095584    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:22.067206 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.067172    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:22.480083 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.480038    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:22.749148 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749056    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"]
Sep 06 05:23:22.749548 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749531    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1832b4be-8f46-425b-a040-e0b59f182a5f" containerName="payload-processing"
Sep 06 05:23:22.749626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749549    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1832b4be-8f46-425b-a040-e0b59f182a5f" containerName="payload-processing"
Sep 06 05:23:22.749626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749563    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2382715c-e2ff-4ef7-983f-0bebcbac420c" containerName="payload-processing"
Sep 06 05:23:22.749626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749569    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2382715c-e2ff-4ef7-983f-0bebcbac420c" containerName="payload-processing"
Sep 06 05:23:22.749626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749576    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerName="istio-proxy"
Sep 06 05:23:22.749626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749582    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerName="istio-proxy"
Sep 06 05:23:22.749626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749591    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43725b3f-a6f4-4f8d-b614-679e019d3cb8" containerName="maas-api"
Sep 06 05:23:22.749626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749596    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="43725b3f-a6f4-4f8d-b614-679e019d3cb8" containerName="maas-api"
Sep 06 05:23:22.749626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749607    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerName="istio-proxy"
Sep 06 05:23:22.749626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749613    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerName="istio-proxy"
Sep 06 05:23:22.749626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749619    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4782f5d5-fa27-41a9-854c-81464aa2d38f" containerName="maas-api"
Sep 06 05:23:22.749626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749624    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4782f5d5-fa27-41a9-854c-81464aa2d38f" containerName="maas-api"
Sep 06 05:23:22.749626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749630    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6225ded-788d-4a89-b327-c1a5d727213c" containerName="payload-pre-processing"
Sep 06 05:23:22.749626 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749635    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6225ded-788d-4a89-b327-c1a5d727213c" containerName="payload-pre-processing"
Sep 06 05:23:22.750155 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749645    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3b9b742-ef50-4aac-90a4-01ebbc38590e" containerName="payload-processing"
Sep 06 05:23:22.750155 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749650    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3b9b742-ef50-4aac-90a4-01ebbc38590e" containerName="payload-processing"
Sep 06 05:23:22.750155 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749709    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2382715c-e2ff-4ef7-983f-0bebcbac420c" containerName="payload-processing"
Sep 06 05:23:22.750155 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749717    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerName="istio-proxy"
Sep 06 05:23:22.750155 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749723    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3b9b742-ef50-4aac-90a4-01ebbc38590e" containerName="payload-processing"
Sep 06 05:23:22.750155 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749730    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee44c28c-f2b1-4e58-a413-6fa792dcf4e9" containerName="istio-proxy"
Sep 06 05:23:22.750155 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749738    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="43725b3f-a6f4-4f8d-b614-679e019d3cb8" containerName="maas-api"
Sep 06 05:23:22.750155 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749743    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4782f5d5-fa27-41a9-854c-81464aa2d38f" containerName="maas-api"
Sep 06 05:23:22.750155 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749748    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6225ded-788d-4a89-b327-c1a5d727213c" containerName="payload-pre-processing"
Sep 06 05:23:22.750155 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.749755    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1832b4be-8f46-425b-a040-e0b59f182a5f" containerName="payload-processing"
Sep 06 05:23:22.752411 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.752394    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.754542 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.754517    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-41fda0-openshift-default-dockercfg-ptqtx\""
Sep 06 05:23:22.765278 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.765257    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"]
Sep 06 05:23:22.777857 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.777829    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:22.847205 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.847167    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-token\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.847401 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.847214    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxqcr\" (UniqueName: \"kubernetes.io/projected/38ef5202-290f-4b76-b7b2-4f1e3fbab725-kube-api-access-nxqcr\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.847401 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.847298    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-credential-socket\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.847401 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.847316    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-podinfo\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.847401 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.847335    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-envoy\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.847401 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.847352    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-workload-certs\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.847685 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.847470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-workload-socket\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.847685 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.847510    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istiod-ca-cert\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.847685 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.847560    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-data\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.948317 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.948277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-credential-socket\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.948317 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.948319    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-podinfo\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.948567 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.948341    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-envoy\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.948567 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.948363    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-workload-certs\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.948567 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.948404    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-workload-socket\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.948567 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.948423    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istiod-ca-cert\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.948567 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.948442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-data\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.948567 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.948483    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-token\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.948567 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.948539    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxqcr\" (UniqueName: \"kubernetes.io/projected/38ef5202-290f-4b76-b7b2-4f1e3fbab725-kube-api-access-nxqcr\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.948940 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.948710    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-credential-socket\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.948940 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.948790    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-workload-certs\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.949087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.948989    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-data\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.949087 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.949062    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-workload-socket\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.949307 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.949284    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istiod-ca-cert\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.950697 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.950673    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-envoy\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.950794 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.950771    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-podinfo\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.955688 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.955668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxqcr\" (UniqueName: \"kubernetes.io/projected/38ef5202-290f-4b76-b7b2-4f1e3fbab725-kube-api-access-nxqcr\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:22.956114 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:22.956091    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-token\") pod \"e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") " pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:23.039671 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:23.039584    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:23.064135 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:23.064094    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:23.343201 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:23.343173    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"]
Sep 06 05:23:23.345214 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:23:23.345183    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38ef5202_290f_4b76_b7b2_4f1e3fbab725.slice/crio-e027abbd1c5ed5d3ee6d441248d609b22bdd77d032db4a51f28fb265001554c2 WatchSource:0}: Error finding container e027abbd1c5ed5d3ee6d441248d609b22bdd77d032db4a51f28fb265001554c2: Status 404 returned error can't find the container with id e027abbd1c5ed5d3ee6d441248d609b22bdd77d032db4a51f28fb265001554c2
Sep 06 05:23:23.347611 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:23.347577    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:23:23.347707 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:23.347641    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:23:23.347707 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:23.347669    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:23:23.866826 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:23.866786    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm" event={"ID":"38ef5202-290f-4b76-b7b2-4f1e3fbab725","Type":"ContainerStarted","Data":"6373db5d7adb84217282b00dda42abbfdfdf429d0910a05b88eb42eca3b28c4f"}
Sep 06 05:23:23.867211 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:23.866846    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm" event={"ID":"38ef5202-290f-4b76-b7b2-4f1e3fbab725","Type":"ContainerStarted","Data":"e027abbd1c5ed5d3ee6d441248d609b22bdd77d032db4a51f28fb265001554c2"}
Sep 06 05:23:23.885756 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:23.885703    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm" podStartSLOduration=1.885684506 podStartE2EDuration="1.885684506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:23:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:23:23.883784073 +0000 UTC m=+1671.253239340" watchObservedRunningTime="2026-09-06 05:23:23.885684506 +0000 UTC m=+1671.255139753"
Sep 06 05:23:24.064283 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:24.064246    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:24.069070 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:24.069046    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:24.872287 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:24.872257    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:24.873264 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:24.873245    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:25.851604 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:25.851572    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:25.952602 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:25.952573    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:27.044991 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.044950    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"]
Sep 06 05:23:27.047293 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.047277    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:27.049400 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.049385    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-41fda0-serving-cert\""
Sep 06 05:23:27.056426 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.056404    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"]
Sep 06 05:23:27.089476 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.089444    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f7bf25-c781-476a-9a9a-8a1c95358441-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:27.089668 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.089548    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-metrics-certs\") pod \"maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:27.089668 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.089596    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-maas-api-tls\") pod \"maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:27.089668 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.089636    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trrdv\" (UniqueName: \"kubernetes.io/projected/f1f7bf25-c781-476a-9a9a-8a1c95358441-kube-api-access-trrdv\") pod \"maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:27.146517 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.146484    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:27.190653 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.190602    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f7bf25-c781-476a-9a9a-8a1c95358441-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:27.190854 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.190673    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-metrics-certs\") pod \"maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:27.190854 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.190704    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-maas-api-tls\") pod \"maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:27.190854 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.190729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trrdv\" (UniqueName: \"kubernetes.io/projected/f1f7bf25-c781-476a-9a9a-8a1c95358441-kube-api-access-trrdv\") pod \"maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:27.191064 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:23:27.190862    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-serving-cert: secret "maas-api-e2e-ipp-a-41fda0-serving-cert" not found
Sep 06 05:23:27.191064 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:23:27.190968    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-maas-api-tls podName:f1f7bf25-c781-476a-9a9a-8a1c95358441 nodeName:}" failed. No retries permitted until 2026-09-06 05:23:27.690942996 +0000 UTC m=+1675.060398235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-maas-api-tls") pod "maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz" (UID: "f1f7bf25-c781-476a-9a9a-8a1c95358441") : secret "maas-api-e2e-ipp-a-41fda0-serving-cert" not found
Sep 06 05:23:27.191427 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.191404    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f7bf25-c781-476a-9a9a-8a1c95358441-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:27.193408 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.193378    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-metrics-certs\") pod \"maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:27.201300 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.201270    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trrdv\" (UniqueName: \"kubernetes.io/projected/f1f7bf25-c781-476a-9a9a-8a1c95358441-kube-api-access-trrdv\") pod \"maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:27.246227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.246189    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:27.304189 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.304110    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:27.575555 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.575466    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:27.672891 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.672850    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:27.695487 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.695451    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-maas-api-tls\") pod \"maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:27.698247 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.698213    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-maas-api-tls\") pod \"maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:27.958794 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:27.958760    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:28.230278 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:28.230248    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"]
Sep 06 05:23:28.231544 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:23:28.231516    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1f7bf25_c781_476a_9a9a_8a1c95358441.slice/crio-81eeda9f31dcebbf008a7ec5038a0e30a718453c73bc215784eea5e8c7917102 WatchSource:0}: Error finding container 81eeda9f31dcebbf008a7ec5038a0e30a718453c73bc215784eea5e8c7917102: Status 404 returned error can't find the container with id 81eeda9f31dcebbf008a7ec5038a0e30a718453c73bc215784eea5e8c7917102
Sep 06 05:23:28.897049 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:28.896933    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz" event={"ID":"f1f7bf25-c781-476a-9a9a-8a1c95358441","Type":"ContainerStarted","Data":"d566e8632fcd80ff072d41ef53056d93a32e3b5ca34003bed3e99406f9f6906f"}
Sep 06 05:23:28.897049 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:28.896984    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz" event={"ID":"f1f7bf25-c781-476a-9a9a-8a1c95358441","Type":"ContainerStarted","Data":"81eeda9f31dcebbf008a7ec5038a0e30a718453c73bc215784eea5e8c7917102"}
Sep 06 05:23:28.897259 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:28.897135    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:28.911068 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:28.910989    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz" podStartSLOduration=1.516963422 podStartE2EDuration="1.910975196s" totalImagesPullingTime="394.011774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:23:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:23:28.910940806 +0000 UTC m=+1676.280396051" watchObservedRunningTime="2026-09-06 05:23:28.910975196 +0000 UTC m=+1676.280430442"
Sep 06 05:23:34.915311 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:34.915276    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:23:40.914179 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:40.914140    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 05:23:40.914572 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:40.914216    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm" podUID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 05:23:41.972103 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:41.972065    2578 generic.go:356] "Generic (PLEG): container finished" podID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerID="6373db5d7adb84217282b00dda42abbfdfdf429d0910a05b88eb42eca3b28c4f" exitCode=0
Sep 06 05:23:41.972537 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:41.972116    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm" event={"ID":"38ef5202-290f-4b76-b7b2-4f1e3fbab725","Type":"ContainerDied","Data":"6373db5d7adb84217282b00dda42abbfdfdf429d0910a05b88eb42eca3b28c4f"}
Sep 06 05:23:41.972537 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:41.972507    2578 scope.go:117] "RemoveContainer" containerID="6373db5d7adb84217282b00dda42abbfdfdf429d0910a05b88eb42eca3b28c4f"
Sep 06 05:23:41.973563 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:41.973531    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:23:41.973671 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:41.973612    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:23:41.973671 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:41.973660    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 06 05:23:42.980551 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:42.980515    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm" event={"ID":"38ef5202-290f-4b76-b7b2-4f1e3fbab725","Type":"ContainerStarted","Data":"fe8d20a37f4072f436e2106b1b8ea7be04e32a73e7cb036a68f84918f37f1d31"}
Sep 06 05:23:43.065083 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:43.065050    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:43.065311 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:43.065287    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Sep 06 05:23:43.065398 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:43.065353    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm" podUID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Sep 06 05:23:44.064957 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:44.064922    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Sep 06 05:23:44.065393 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:44.064980    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm" podUID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Sep 06 05:23:45.064796 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:45.064760    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Sep 06 05:23:45.064974 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:45.064828    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm" podUID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Sep 06 05:23:46.064703 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:46.064667    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Sep 06 05:23:46.064889 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:46.064722    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm" podUID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Sep 06 05:23:47.065423 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:47.065386    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Sep 06 05:23:47.065801 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:47.065441    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm" podUID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Sep 06 05:23:48.068765 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:48.068734    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:48.069141 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:48.069116    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:48.069836 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:48.069820    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:23:50.693757 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:50.693725    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:51.333611 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:51.333580    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:51.639350 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:51.639323    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:51.950286 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:51.950195    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:54.517317 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:54.517282    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:54.657547 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:54.657512    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:55.622973 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:55.622946    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:55.723611 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:55.723573    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:55.779764 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:55.779723    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:56.087042 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.086959    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:56.193736 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.193697    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:23:56.231584 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.231547    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"]
Sep 06 05:23:56.237301 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.237277    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"
Sep 06 05:23:56.239528 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.239507    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-22321a-dockercfg-mp5xx\""
Sep 06 05:23:56.239648 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.239532    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-22321a\""
Sep 06 05:23:56.243517 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.243490    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"]
Sep 06 05:23:56.384172 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.384134    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74lch\" (UniqueName: \"kubernetes.io/projected/8a5e5e07-abda-480c-affd-640fba2c2b95-kube-api-access-74lch\") pod \"payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl\" (UID: \"8a5e5e07-abda-480c-affd-640fba2c2b95\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"
Sep 06 05:23:56.384341 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.384236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a5e5e07-abda-480c-affd-640fba2c2b95-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl\" (UID: \"8a5e5e07-abda-480c-affd-640fba2c2b95\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"
Sep 06 05:23:56.485659 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.485620    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74lch\" (UniqueName: \"kubernetes.io/projected/8a5e5e07-abda-480c-affd-640fba2c2b95-kube-api-access-74lch\") pod \"payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl\" (UID: \"8a5e5e07-abda-480c-affd-640fba2c2b95\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"
Sep 06 05:23:56.485821 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.485696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a5e5e07-abda-480c-affd-640fba2c2b95-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl\" (UID: \"8a5e5e07-abda-480c-affd-640fba2c2b95\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"
Sep 06 05:23:56.486409 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.486389    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a5e5e07-abda-480c-affd-640fba2c2b95-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl\" (UID: \"8a5e5e07-abda-480c-affd-640fba2c2b95\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"
Sep 06 05:23:56.494422 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.494391    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74lch\" (UniqueName: \"kubernetes.io/projected/8a5e5e07-abda-480c-affd-640fba2c2b95-kube-api-access-74lch\") pod \"payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl\" (UID: \"8a5e5e07-abda-480c-affd-640fba2c2b95\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"
Sep 06 05:23:56.548800 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.548762    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"
Sep 06 05:23:56.823425 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:56.820680    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"]
Sep 06 05:23:56.824764 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:23:56.824734    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a5e5e07_abda_480c_affd_640fba2c2b95.slice/crio-b1f8513158481c28026a41f7fef86bd46d21b4e9cc3717cdadc6261ed10fc714 WatchSource:0}: Error finding container b1f8513158481c28026a41f7fef86bd46d21b4e9cc3717cdadc6261ed10fc714: Status 404 returned error can't find the container with id b1f8513158481c28026a41f7fef86bd46d21b4e9cc3717cdadc6261ed10fc714
Sep 06 05:23:57.067093 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:57.066787    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl" event={"ID":"8a5e5e07-abda-480c-affd-640fba2c2b95","Type":"ContainerStarted","Data":"201527f91142af63c5edc871b84f1407f19a872f93638a2084e85faa8b96f4c6"}
Sep 06 05:23:57.067093 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:57.066851    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl" event={"ID":"8a5e5e07-abda-480c-affd-640fba2c2b95","Type":"ContainerStarted","Data":"b1f8513158481c28026a41f7fef86bd46d21b4e9cc3717cdadc6261ed10fc714"}
Sep 06 05:23:57.067093 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:57.066900    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"
Sep 06 05:23:57.083670 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:23:57.083564    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl" podStartSLOduration=1.083549124 podStartE2EDuration="1.083549124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:23:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:23:57.080597958 +0000 UTC m=+1704.450053199" watchObservedRunningTime="2026-09-06 05:23:57.083549124 +0000 UTC m=+1704.453004395"
Sep 06 05:24:01.398751 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:24:01.398715    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:24:01.450994 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:24:01.450959    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:24:01.618227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:24:01.618189    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:24:01.668237 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:24:01.668139    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:24:08.073963 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:24:08.073930    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"
Sep 06 05:24:19.567127 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:24:19.567093    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:24:24.843162 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:24:24.843117    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-b5x42_2c73f38f-cd1d-4b43-9f88-e7a06578fdfc/payload-processing/0.log"
Sep 06 05:24:24.966872 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:24:24.966836    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl_8a5e5e07-abda-480c-affd-640fba2c2b95/payload-processing/0.log"
Sep 06 05:24:38.214959 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:24:38.214926    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:24:42.180579 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:24:42.180546    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:08.216435 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:08.216396    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-b5x42_2c73f38f-cd1d-4b43-9f88-e7a06578fdfc/payload-processing/0.log"
Sep 06 05:25:08.349301 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:08.349270    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl_8a5e5e07-abda-480c-affd-640fba2c2b95/payload-processing/0.log"
Sep 06 05:25:08.626296 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:08.626265    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:09.047289 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:09.047211    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:09.176749 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:09.176715    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:11.839828 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:11.839781    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:11.938443 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:11.938399    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:13.087904 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.087873    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:13.189160 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.189121    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:13.477056 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.477001    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:13.638955 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.638926    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:13.735340 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.735252    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:13.779920 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.779885    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"]
Sep 06 05:25:13.783510 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.783492    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"
Sep 06 05:25:13.785903 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.785878    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-c0aaed\""
Sep 06 05:25:13.786038 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.785880    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-c0aaed-dockercfg-c87vn\""
Sep 06 05:25:13.790786 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.790641    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"]
Sep 06 05:25:13.850892 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.850843    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9thgj\" (UniqueName: \"kubernetes.io/projected/365b22df-580d-41d2-a343-c861626edcb1-kube-api-access-9thgj\") pod \"payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw\" (UID: \"365b22df-580d-41d2-a343-c861626edcb1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"
Sep 06 05:25:13.851103 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.850951    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/365b22df-580d-41d2-a343-c861626edcb1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw\" (UID: \"365b22df-580d-41d2-a343-c861626edcb1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"
Sep 06 05:25:13.952098 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.952054    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9thgj\" (UniqueName: \"kubernetes.io/projected/365b22df-580d-41d2-a343-c861626edcb1-kube-api-access-9thgj\") pod \"payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw\" (UID: \"365b22df-580d-41d2-a343-c861626edcb1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"
Sep 06 05:25:13.952300 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.952158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/365b22df-580d-41d2-a343-c861626edcb1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw\" (UID: \"365b22df-580d-41d2-a343-c861626edcb1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"
Sep 06 05:25:13.952859 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.952839    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/365b22df-580d-41d2-a343-c861626edcb1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw\" (UID: \"365b22df-580d-41d2-a343-c861626edcb1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"
Sep 06 05:25:13.960491 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:13.960465    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9thgj\" (UniqueName: \"kubernetes.io/projected/365b22df-580d-41d2-a343-c861626edcb1-kube-api-access-9thgj\") pod \"payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw\" (UID: \"365b22df-580d-41d2-a343-c861626edcb1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"
Sep 06 05:25:14.098416 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:14.098331    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"
Sep 06 05:25:14.367899 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:14.367871    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"]
Sep 06 05:25:14.370353 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:25:14.370327    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod365b22df_580d_41d2_a343_c861626edcb1.slice/crio-61115aa78bbeef193ff393dbf7ad80513d59b25febdf83798928400a5bcd4389 WatchSource:0}: Error finding container 61115aa78bbeef193ff393dbf7ad80513d59b25febdf83798928400a5bcd4389: Status 404 returned error can't find the container with id 61115aa78bbeef193ff393dbf7ad80513d59b25febdf83798928400a5bcd4389
Sep 06 05:25:14.480879 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:14.480847    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw" event={"ID":"365b22df-580d-41d2-a343-c861626edcb1","Type":"ContainerStarted","Data":"ac09d02db8832d13d431a0e927f107ca6803fb84e28d7f95b78a8d9dfa01fa32"}
Sep 06 05:25:14.481079 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:14.480896    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw" event={"ID":"365b22df-580d-41d2-a343-c861626edcb1","Type":"ContainerStarted","Data":"61115aa78bbeef193ff393dbf7ad80513d59b25febdf83798928400a5bcd4389"}
Sep 06 05:25:14.481079 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:14.481042    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"
Sep 06 05:25:14.497101 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:14.497054    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw" podStartSLOduration=1.497038752 podStartE2EDuration="1.497038752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:25:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:25:14.495519351 +0000 UTC m=+1781.864974600" watchObservedRunningTime="2026-09-06 05:25:14.497038752 +0000 UTC m=+1781.866493997"
Sep 06 05:25:25.488615 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:25.488579    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"
Sep 06 05:25:36.877481 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:36.877431    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:43.399786 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.399748    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:43.423487 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.423457    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"]
Sep 06 05:25:43.423706 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.423680    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw" podUID="365b22df-580d-41d2-a343-c861626edcb1" containerName="payload-processing" containerID="cri-o://ac09d02db8832d13d431a0e927f107ca6803fb84e28d7f95b78a8d9dfa01fa32" gracePeriod=30
Sep 06 05:25:43.500345 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.500311    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:43.573842 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.573810    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:43.626067 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.626038    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:43.637800 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.637767    2578 generic.go:356] "Generic (PLEG): container finished" podID="365b22df-580d-41d2-a343-c861626edcb1" containerID="ac09d02db8832d13d431a0e927f107ca6803fb84e28d7f95b78a8d9dfa01fa32" exitCode=0
Sep 06 05:25:43.637951 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.637832    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw" event={"ID":"365b22df-580d-41d2-a343-c861626edcb1","Type":"ContainerDied","Data":"ac09d02db8832d13d431a0e927f107ca6803fb84e28d7f95b78a8d9dfa01fa32"}
Sep 06 05:25:43.697257 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.697231    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"
Sep 06 05:25:43.849645 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.849608    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9thgj\" (UniqueName: \"kubernetes.io/projected/365b22df-580d-41d2-a343-c861626edcb1-kube-api-access-9thgj\") pod \"365b22df-580d-41d2-a343-c861626edcb1\" (UID: \"365b22df-580d-41d2-a343-c861626edcb1\") "
Sep 06 05:25:43.849865 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.849666    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/365b22df-580d-41d2-a343-c861626edcb1-plugins-config-volume\") pod \"365b22df-580d-41d2-a343-c861626edcb1\" (UID: \"365b22df-580d-41d2-a343-c861626edcb1\") "
Sep 06 05:25:43.850127 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.850097    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/365b22df-580d-41d2-a343-c861626edcb1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "365b22df-580d-41d2-a343-c861626edcb1" (UID: "365b22df-580d-41d2-a343-c861626edcb1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:25:43.851820 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.851801    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/365b22df-580d-41d2-a343-c861626edcb1-kube-api-access-9thgj" (OuterVolumeSpecName: "kube-api-access-9thgj") pod "365b22df-580d-41d2-a343-c861626edcb1" (UID: "365b22df-580d-41d2-a343-c861626edcb1"). InnerVolumeSpecName "kube-api-access-9thgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:25:43.951077 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.950924    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9thgj\" (UniqueName: \"kubernetes.io/projected/365b22df-580d-41d2-a343-c861626edcb1-kube-api-access-9thgj\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:25:43.951077 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.950966    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/365b22df-580d-41d2-a343-c861626edcb1-plugins-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:25:43.959183 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:43.959146    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:44.644344 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:44.644315    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"
Sep 06 05:25:44.644723 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:44.644363    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw" event={"ID":"365b22df-580d-41d2-a343-c861626edcb1","Type":"ContainerDied","Data":"61115aa78bbeef193ff393dbf7ad80513d59b25febdf83798928400a5bcd4389"}
Sep 06 05:25:44.644723 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:44.644416    2578 scope.go:117] "RemoveContainer" containerID="ac09d02db8832d13d431a0e927f107ca6803fb84e28d7f95b78a8d9dfa01fa32"
Sep 06 05:25:44.666115 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:44.666088    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"]
Sep 06 05:25:44.669214 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:44.669193    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-c0aaed-6bf6fb9bcf-h59tw"]
Sep 06 05:25:45.159075 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:45.159040    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="365b22df-580d-41d2-a343-c861626edcb1" path="/var/lib/kubelet/pods/365b22df-580d-41d2-a343-c861626edcb1/volumes"
Sep 06 05:25:49.574663 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:49.574629    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:49.627510 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:49.627474    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:49.845748 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:49.845663    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:25:53.587047 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:25:53.586994    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:26:08.921065 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:08.921033    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"]
Sep 06 05:26:08.921493 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:08.921217    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz" podUID="f1f7bf25-c781-476a-9a9a-8a1c95358441" containerName="maas-api" containerID="cri-o://d566e8632fcd80ff072d41ef53056d93a32e3b5ca34003bed3e99406f9f6906f" gracePeriod=30
Sep 06 05:26:08.949080 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:08.949050    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:26:09.011741 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.011712    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:26:09.069358 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.069323    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:26:09.125078 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.125041    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:26:09.180144 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.180074    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:26:09.293207 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.293166    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f7bf25-c781-476a-9a9a-8a1c95358441-openshift-service-ca\") pod \"f1f7bf25-c781-476a-9a9a-8a1c95358441\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") "
Sep 06 05:26:09.293397 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.293299    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-maas-api-tls\") pod \"f1f7bf25-c781-476a-9a9a-8a1c95358441\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") "
Sep 06 05:26:09.293397 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.293345    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trrdv\" (UniqueName: \"kubernetes.io/projected/f1f7bf25-c781-476a-9a9a-8a1c95358441-kube-api-access-trrdv\") pod \"f1f7bf25-c781-476a-9a9a-8a1c95358441\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") "
Sep 06 05:26:09.293397 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.293374    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-metrics-certs\") pod \"f1f7bf25-c781-476a-9a9a-8a1c95358441\" (UID: \"f1f7bf25-c781-476a-9a9a-8a1c95358441\") "
Sep 06 05:26:09.293566 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.293530    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1f7bf25-c781-476a-9a9a-8a1c95358441-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f1f7bf25-c781-476a-9a9a-8a1c95358441" (UID: "f1f7bf25-c781-476a-9a9a-8a1c95358441"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:26:09.293822 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.293690    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f7bf25-c781-476a-9a9a-8a1c95358441-openshift-service-ca\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:26:09.296196 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.296169    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f1f7bf25-c781-476a-9a9a-8a1c95358441" (UID: "f1f7bf25-c781-476a-9a9a-8a1c95358441"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:26:09.296664 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.296626    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f1f7bf25-c781-476a-9a9a-8a1c95358441" (UID: "f1f7bf25-c781-476a-9a9a-8a1c95358441"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:26:09.296764 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.296734    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1f7bf25-c781-476a-9a9a-8a1c95358441-kube-api-access-trrdv" (OuterVolumeSpecName: "kube-api-access-trrdv") pod "f1f7bf25-c781-476a-9a9a-8a1c95358441" (UID: "f1f7bf25-c781-476a-9a9a-8a1c95358441"). InnerVolumeSpecName "kube-api-access-trrdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:26:09.394363 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.394324    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-maas-api-tls\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:26:09.394363 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.394361    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trrdv\" (UniqueName: \"kubernetes.io/projected/f1f7bf25-c781-476a-9a9a-8a1c95358441-kube-api-access-trrdv\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:26:09.394584 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.394376    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1f7bf25-c781-476a-9a9a-8a1c95358441-metrics-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:26:09.501085 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.500959    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:26:09.784441 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.784351    2578 generic.go:356] "Generic (PLEG): container finished" podID="f1f7bf25-c781-476a-9a9a-8a1c95358441" containerID="d566e8632fcd80ff072d41ef53056d93a32e3b5ca34003bed3e99406f9f6906f" exitCode=0
Sep 06 05:26:09.784441 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.784392    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"
Sep 06 05:26:09.784441 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.784432    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz" event={"ID":"f1f7bf25-c781-476a-9a9a-8a1c95358441","Type":"ContainerDied","Data":"d566e8632fcd80ff072d41ef53056d93a32e3b5ca34003bed3e99406f9f6906f"}
Sep 06 05:26:09.784691 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.784484    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz" event={"ID":"f1f7bf25-c781-476a-9a9a-8a1c95358441","Type":"ContainerDied","Data":"81eeda9f31dcebbf008a7ec5038a0e30a718453c73bc215784eea5e8c7917102"}
Sep 06 05:26:09.784691 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.784507    2578 scope.go:117] "RemoveContainer" containerID="d566e8632fcd80ff072d41ef53056d93a32e3b5ca34003bed3e99406f9f6906f"
Sep 06 05:26:09.795285 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.795268    2578 scope.go:117] "RemoveContainer" containerID="d566e8632fcd80ff072d41ef53056d93a32e3b5ca34003bed3e99406f9f6906f"
Sep 06 05:26:09.795550 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:26:09.795535    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d566e8632fcd80ff072d41ef53056d93a32e3b5ca34003bed3e99406f9f6906f\": container with ID starting with d566e8632fcd80ff072d41ef53056d93a32e3b5ca34003bed3e99406f9f6906f not found: ID does not exist" containerID="d566e8632fcd80ff072d41ef53056d93a32e3b5ca34003bed3e99406f9f6906f"
Sep 06 05:26:09.795594 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.795557    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d566e8632fcd80ff072d41ef53056d93a32e3b5ca34003bed3e99406f9f6906f"} err="failed to get container status \"d566e8632fcd80ff072d41ef53056d93a32e3b5ca34003bed3e99406f9f6906f\": rpc error: code = NotFound desc = could not find container \"d566e8632fcd80ff072d41ef53056d93a32e3b5ca34003bed3e99406f9f6906f\": container with ID starting with d566e8632fcd80ff072d41ef53056d93a32e3b5ca34003bed3e99406f9f6906f not found: ID does not exist"
Sep 06 05:26:09.805638 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.805616    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"]
Sep 06 05:26:09.809153 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:09.809132    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-41fda0-7c6667cf4-8sdrz"]
Sep 06 05:26:11.154869 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:11.154834    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1f7bf25-c781-476a-9a9a-8a1c95358441" path="/var/lib/kubelet/pods/f1f7bf25-c781-476a-9a9a-8a1c95358441/volumes"
Sep 06 05:26:14.573128 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:14.573091    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:26:50.710311 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:50.710271    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:26:51.618321 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:51.618284    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:26:55.879416 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:55.879381    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:26:57.203359 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:57.203326    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:26:57.302773 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:57.302738    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:26:57.447771 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:57.447735    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"]
Sep 06 05:26:57.447973 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:57.447946    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm" podUID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerName="istio-proxy" containerID="cri-o://fe8d20a37f4072f436e2106b1b8ea7be04e32a73e7cb036a68f84918f37f1d31" gracePeriod=30
Sep 06 05:26:57.472319 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:57.472244    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:26:58.069245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:26:58.069197    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:02.712996 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.712930    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:27:02.813252 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.813220    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-credential-socket\") pod \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") "
Sep 06 05:27:02.813252 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.813258    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-envoy\") pod \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") "
Sep 06 05:27:02.813505 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.813284    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istiod-ca-cert\") pod \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") "
Sep 06 05:27:02.813505 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.813347    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-podinfo\") pod \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") "
Sep 06 05:27:02.813505 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.813394    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-token\") pod \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") "
Sep 06 05:27:02.813505 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.813449    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-workload-certs\") pod \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") "
Sep 06 05:27:02.813505 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.813487    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxqcr\" (UniqueName: \"kubernetes.io/projected/38ef5202-290f-4b76-b7b2-4f1e3fbab725-kube-api-access-nxqcr\") pod \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") "
Sep 06 05:27:02.813752 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.813523    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-data\") pod \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") "
Sep 06 05:27:02.813752 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.813568    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-workload-socket\") pod \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\" (UID: \"38ef5202-290f-4b76-b7b2-4f1e3fbab725\") "
Sep 06 05:27:02.813752 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.813584    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "38ef5202-290f-4b76-b7b2-4f1e3fbab725" (UID: "38ef5202-290f-4b76-b7b2-4f1e3fbab725"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:02.813752 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.813725    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "38ef5202-290f-4b76-b7b2-4f1e3fbab725" (UID: "38ef5202-290f-4b76-b7b2-4f1e3fbab725"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:27:02.813956 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.813846    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "38ef5202-290f-4b76-b7b2-4f1e3fbab725" (UID: "38ef5202-290f-4b76-b7b2-4f1e3fbab725"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:02.813956 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.813907    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "38ef5202-290f-4b76-b7b2-4f1e3fbab725" (UID: "38ef5202-290f-4b76-b7b2-4f1e3fbab725"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:02.814100 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.814071    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-workload-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:02.814100 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.814088    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-workload-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:02.814248 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.814099    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-credential-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:02.814248 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.814113    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istiod-ca-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:02.814248 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.814116    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-data" (OuterVolumeSpecName: "istio-data") pod "38ef5202-290f-4b76-b7b2-4f1e3fbab725" (UID: "38ef5202-290f-4b76-b7b2-4f1e3fbab725"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:02.815577 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.815546    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "38ef5202-290f-4b76-b7b2-4f1e3fbab725" (UID: "38ef5202-290f-4b76-b7b2-4f1e3fbab725"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:02.815738 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.815716    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38ef5202-290f-4b76-b7b2-4f1e3fbab725-kube-api-access-nxqcr" (OuterVolumeSpecName: "kube-api-access-nxqcr") pod "38ef5202-290f-4b76-b7b2-4f1e3fbab725" (UID: "38ef5202-290f-4b76-b7b2-4f1e3fbab725"). InnerVolumeSpecName "kube-api-access-nxqcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:02.815952 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.815931    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-token" (OuterVolumeSpecName: "istio-token") pod "38ef5202-290f-4b76-b7b2-4f1e3fbab725" (UID: "38ef5202-290f-4b76-b7b2-4f1e3fbab725"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:02.815952 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.815938    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "38ef5202-290f-4b76-b7b2-4f1e3fbab725" (UID: "38ef5202-290f-4b76-b7b2-4f1e3fbab725"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:27:02.914908 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.914876    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-podinfo\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:02.914908 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.914906    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-token\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:02.914908 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.914916    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxqcr\" (UniqueName: \"kubernetes.io/projected/38ef5202-290f-4b76-b7b2-4f1e3fbab725-kube-api-access-nxqcr\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:02.915245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.914927    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-data\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:02.915245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:02.914936    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38ef5202-290f-4b76-b7b2-4f1e3fbab725-istio-envoy\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:03.073450 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:03.073360    2578 generic.go:356] "Generic (PLEG): container finished" podID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerID="fe8d20a37f4072f436e2106b1b8ea7be04e32a73e7cb036a68f84918f37f1d31" exitCode=0
Sep 06 05:27:03.073450 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:03.073409    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"
Sep 06 05:27:03.073636 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:03.073450    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm" event={"ID":"38ef5202-290f-4b76-b7b2-4f1e3fbab725","Type":"ContainerDied","Data":"fe8d20a37f4072f436e2106b1b8ea7be04e32a73e7cb036a68f84918f37f1d31"}
Sep 06 05:27:03.073636 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:03.073504    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm" event={"ID":"38ef5202-290f-4b76-b7b2-4f1e3fbab725","Type":"ContainerDied","Data":"e027abbd1c5ed5d3ee6d441248d609b22bdd77d032db4a51f28fb265001554c2"}
Sep 06 05:27:03.073636 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:03.073527    2578 scope.go:117] "RemoveContainer" containerID="fe8d20a37f4072f436e2106b1b8ea7be04e32a73e7cb036a68f84918f37f1d31"
Sep 06 05:27:03.082594 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:03.082576    2578 scope.go:117] "RemoveContainer" containerID="6373db5d7adb84217282b00dda42abbfdfdf429d0910a05b88eb42eca3b28c4f"
Sep 06 05:27:03.090546 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:03.090529    2578 scope.go:117] "RemoveContainer" containerID="fe8d20a37f4072f436e2106b1b8ea7be04e32a73e7cb036a68f84918f37f1d31"
Sep 06 05:27:03.090850 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:27:03.090832    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe8d20a37f4072f436e2106b1b8ea7be04e32a73e7cb036a68f84918f37f1d31\": container with ID starting with fe8d20a37f4072f436e2106b1b8ea7be04e32a73e7cb036a68f84918f37f1d31 not found: ID does not exist" containerID="fe8d20a37f4072f436e2106b1b8ea7be04e32a73e7cb036a68f84918f37f1d31"
Sep 06 05:27:03.090898 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:03.090860    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe8d20a37f4072f436e2106b1b8ea7be04e32a73e7cb036a68f84918f37f1d31"} err="failed to get container status \"fe8d20a37f4072f436e2106b1b8ea7be04e32a73e7cb036a68f84918f37f1d31\": rpc error: code = NotFound desc = could not find container \"fe8d20a37f4072f436e2106b1b8ea7be04e32a73e7cb036a68f84918f37f1d31\": container with ID starting with fe8d20a37f4072f436e2106b1b8ea7be04e32a73e7cb036a68f84918f37f1d31 not found: ID does not exist"
Sep 06 05:27:03.090898 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:03.090879    2578 scope.go:117] "RemoveContainer" containerID="6373db5d7adb84217282b00dda42abbfdfdf429d0910a05b88eb42eca3b28c4f"
Sep 06 05:27:03.091222 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:27:03.091194    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6373db5d7adb84217282b00dda42abbfdfdf429d0910a05b88eb42eca3b28c4f\": container with ID starting with 6373db5d7adb84217282b00dda42abbfdfdf429d0910a05b88eb42eca3b28c4f not found: ID does not exist" containerID="6373db5d7adb84217282b00dda42abbfdfdf429d0910a05b88eb42eca3b28c4f"
Sep 06 05:27:03.091323 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:03.091228    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6373db5d7adb84217282b00dda42abbfdfdf429d0910a05b88eb42eca3b28c4f"} err="failed to get container status \"6373db5d7adb84217282b00dda42abbfdfdf429d0910a05b88eb42eca3b28c4f\": rpc error: code = NotFound desc = could not find container \"6373db5d7adb84217282b00dda42abbfdfdf429d0910a05b88eb42eca3b28c4f\": container with ID starting with 6373db5d7adb84217282b00dda42abbfdfdf429d0910a05b88eb42eca3b28c4f not found: ID does not exist"
Sep 06 05:27:03.094076 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:03.094054    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"]
Sep 06 05:27:03.098326 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:03.098306    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-41fda0-openshift-default-6f9b6f7c58-t7dfm"]
Sep 06 05:27:03.153344 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:03.153320    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" path="/var/lib/kubelet/pods/38ef5202-290f-4b76-b7b2-4f1e3fbab725/volumes"
Sep 06 05:27:08.227270 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:08.227231    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:08.251731 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:08.251700    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"]
Sep 06 05:27:08.251926 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:08.251902    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl" podUID="8a5e5e07-abda-480c-affd-640fba2c2b95" containerName="payload-processing" containerID="cri-o://201527f91142af63c5edc871b84f1407f19a872f93638a2084e85faa8b96f4c6" gracePeriod=30
Sep 06 05:27:08.327890 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:08.327859    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:08.401353 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:08.401318    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:08.452843 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:08.452813    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:08.524590 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:08.524563    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"
Sep 06 05:27:08.569186 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:08.569154    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a5e5e07-abda-480c-affd-640fba2c2b95-plugins-config-volume\") pod \"8a5e5e07-abda-480c-affd-640fba2c2b95\" (UID: \"8a5e5e07-abda-480c-affd-640fba2c2b95\") "
Sep 06 05:27:08.569323 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:08.569199    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74lch\" (UniqueName: \"kubernetes.io/projected/8a5e5e07-abda-480c-affd-640fba2c2b95-kube-api-access-74lch\") pod \"8a5e5e07-abda-480c-affd-640fba2c2b95\" (UID: \"8a5e5e07-abda-480c-affd-640fba2c2b95\") "
Sep 06 05:27:08.569492 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:08.569471    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a5e5e07-abda-480c-affd-640fba2c2b95-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a5e5e07-abda-480c-affd-640fba2c2b95" (UID: "8a5e5e07-abda-480c-affd-640fba2c2b95"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:27:08.571289 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:08.571269    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a5e5e07-abda-480c-affd-640fba2c2b95-kube-api-access-74lch" (OuterVolumeSpecName: "kube-api-access-74lch") pod "8a5e5e07-abda-480c-affd-640fba2c2b95" (UID: "8a5e5e07-abda-480c-affd-640fba2c2b95"). InnerVolumeSpecName "kube-api-access-74lch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:08.670732 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:08.670699    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a5e5e07-abda-480c-affd-640fba2c2b95-plugins-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:08.670732 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:08.670725    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74lch\" (UniqueName: \"kubernetes.io/projected/8a5e5e07-abda-480c-affd-640fba2c2b95-kube-api-access-74lch\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:08.770677 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:08.770559    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:09.109313 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:09.109230    2578 generic.go:356] "Generic (PLEG): container finished" podID="8a5e5e07-abda-480c-affd-640fba2c2b95" containerID="201527f91142af63c5edc871b84f1407f19a872f93638a2084e85faa8b96f4c6" exitCode=0
Sep 06 05:27:09.109313 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:09.109275    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"
Sep 06 05:27:09.109313 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:09.109284    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl" event={"ID":"8a5e5e07-abda-480c-affd-640fba2c2b95","Type":"ContainerDied","Data":"201527f91142af63c5edc871b84f1407f19a872f93638a2084e85faa8b96f4c6"}
Sep 06 05:27:09.109535 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:09.109330    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl" event={"ID":"8a5e5e07-abda-480c-affd-640fba2c2b95","Type":"ContainerDied","Data":"b1f8513158481c28026a41f7fef86bd46d21b4e9cc3717cdadc6261ed10fc714"}
Sep 06 05:27:09.109535 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:09.109354    2578 scope.go:117] "RemoveContainer" containerID="201527f91142af63c5edc871b84f1407f19a872f93638a2084e85faa8b96f4c6"
Sep 06 05:27:09.118646 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:09.118628    2578 scope.go:117] "RemoveContainer" containerID="201527f91142af63c5edc871b84f1407f19a872f93638a2084e85faa8b96f4c6"
Sep 06 05:27:09.118914 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:27:09.118896    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"201527f91142af63c5edc871b84f1407f19a872f93638a2084e85faa8b96f4c6\": container with ID starting with 201527f91142af63c5edc871b84f1407f19a872f93638a2084e85faa8b96f4c6 not found: ID does not exist" containerID="201527f91142af63c5edc871b84f1407f19a872f93638a2084e85faa8b96f4c6"
Sep 06 05:27:09.118954 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:09.118925    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"201527f91142af63c5edc871b84f1407f19a872f93638a2084e85faa8b96f4c6"} err="failed to get container status \"201527f91142af63c5edc871b84f1407f19a872f93638a2084e85faa8b96f4c6\": rpc error: code = NotFound desc = could not find container \"201527f91142af63c5edc871b84f1407f19a872f93638a2084e85faa8b96f4c6\": container with ID starting with 201527f91142af63c5edc871b84f1407f19a872f93638a2084e85faa8b96f4c6 not found: ID does not exist"
Sep 06 05:27:09.136739 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:09.136706    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"]
Sep 06 05:27:09.143165 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:09.143141    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-22321a-57645586df-ldfbl"]
Sep 06 05:27:09.153295 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:09.153274    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a5e5e07-abda-480c-affd-640fba2c2b95" path="/var/lib/kubelet/pods/8a5e5e07-abda-480c-affd-640fba2c2b95/volumes"
Sep 06 05:27:14.369421 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:14.369388    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:14.424600 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:14.424569    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:14.630187 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:14.630106    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:15.067349 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:15.067253    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:24.564367 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:24.564325    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:25.083595 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:25.083561    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:25.111190 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:25.111156    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"]
Sep 06 05:27:25.111488 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:25.111436    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c" podUID="a65b31e0-ed0c-4e66-913a-1f38b47c37db" containerName="payload-processing" containerID="cri-o://e4bd606c9a12d1493ce3a393a19dee3d6f4b55d51a0ae89069c7d222ee58fb09" gracePeriod=30
Sep 06 05:27:25.144951 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:25.144922    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:25.245090 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:25.245058    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:25.326726 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:25.326632    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:25.371754 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:25.371731    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"
Sep 06 05:27:25.429538 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:25.429503    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvz48\" (UniqueName: \"kubernetes.io/projected/a65b31e0-ed0c-4e66-913a-1f38b47c37db-kube-api-access-cvz48\") pod \"a65b31e0-ed0c-4e66-913a-1f38b47c37db\" (UID: \"a65b31e0-ed0c-4e66-913a-1f38b47c37db\") "
Sep 06 05:27:25.429687 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:25.429572    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a65b31e0-ed0c-4e66-913a-1f38b47c37db-plugins-config-volume\") pod \"a65b31e0-ed0c-4e66-913a-1f38b47c37db\" (UID: \"a65b31e0-ed0c-4e66-913a-1f38b47c37db\") "
Sep 06 05:27:25.429952 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:25.429926    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a65b31e0-ed0c-4e66-913a-1f38b47c37db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a65b31e0-ed0c-4e66-913a-1f38b47c37db" (UID: "a65b31e0-ed0c-4e66-913a-1f38b47c37db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:27:25.433391 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:25.433345    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a65b31e0-ed0c-4e66-913a-1f38b47c37db-kube-api-access-cvz48" (OuterVolumeSpecName: "kube-api-access-cvz48") pod "a65b31e0-ed0c-4e66-913a-1f38b47c37db" (UID: "a65b31e0-ed0c-4e66-913a-1f38b47c37db"). InnerVolumeSpecName "kube-api-access-cvz48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:25.530489 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:25.530452    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvz48\" (UniqueName: \"kubernetes.io/projected/a65b31e0-ed0c-4e66-913a-1f38b47c37db-kube-api-access-cvz48\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:25.530489 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:25.530484    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a65b31e0-ed0c-4e66-913a-1f38b47c37db-plugins-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:25.616944 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:25.616857    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:26.204250 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:26.204210    2578 generic.go:356] "Generic (PLEG): container finished" podID="a65b31e0-ed0c-4e66-913a-1f38b47c37db" containerID="e4bd606c9a12d1493ce3a393a19dee3d6f4b55d51a0ae89069c7d222ee58fb09" exitCode=0
Sep 06 05:27:26.204444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:26.204255    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"
Sep 06 05:27:26.204444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:26.204268    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c" event={"ID":"a65b31e0-ed0c-4e66-913a-1f38b47c37db","Type":"ContainerDied","Data":"e4bd606c9a12d1493ce3a393a19dee3d6f4b55d51a0ae89069c7d222ee58fb09"}
Sep 06 05:27:26.204444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:26.204315    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c" event={"ID":"a65b31e0-ed0c-4e66-913a-1f38b47c37db","Type":"ContainerDied","Data":"6c3177542d57ad726ee05305cc493139d74a173e7c33109dc60b776fcc2a963b"}
Sep 06 05:27:26.204444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:26.204340    2578 scope.go:117] "RemoveContainer" containerID="e4bd606c9a12d1493ce3a393a19dee3d6f4b55d51a0ae89069c7d222ee58fb09"
Sep 06 05:27:26.213101 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:26.213084    2578 scope.go:117] "RemoveContainer" containerID="e4bd606c9a12d1493ce3a393a19dee3d6f4b55d51a0ae89069c7d222ee58fb09"
Sep 06 05:27:26.213385 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:27:26.213365    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4bd606c9a12d1493ce3a393a19dee3d6f4b55d51a0ae89069c7d222ee58fb09\": container with ID starting with e4bd606c9a12d1493ce3a393a19dee3d6f4b55d51a0ae89069c7d222ee58fb09 not found: ID does not exist" containerID="e4bd606c9a12d1493ce3a393a19dee3d6f4b55d51a0ae89069c7d222ee58fb09"
Sep 06 05:27:26.213444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:26.213394    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4bd606c9a12d1493ce3a393a19dee3d6f4b55d51a0ae89069c7d222ee58fb09"} err="failed to get container status \"e4bd606c9a12d1493ce3a393a19dee3d6f4b55d51a0ae89069c7d222ee58fb09\": rpc error: code = NotFound desc = could not find container \"e4bd606c9a12d1493ce3a393a19dee3d6f4b55d51a0ae89069c7d222ee58fb09\": container with ID starting with e4bd606c9a12d1493ce3a393a19dee3d6f4b55d51a0ae89069c7d222ee58fb09 not found: ID does not exist"
Sep 06 05:27:26.224173 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:26.224139    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"]
Sep 06 05:27:26.227585 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:26.227562    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-494740-78d988cd47-8wm6c"]
Sep 06 05:27:27.153674 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:27.153640    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a65b31e0-ed0c-4e66-913a-1f38b47c37db" path="/var/lib/kubelet/pods/a65b31e0-ed0c-4e66-913a-1f38b47c37db/volumes"
Sep 06 05:27:31.625544 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:31.625506    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:31.687853 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:31.687828    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:31.864128 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:31.864083    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"]
Sep 06 05:27:31.864389 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:31.864364    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq" podUID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerName="istio-proxy" containerID="cri-o://6a0b642357180461692905f2e3bcc14008d3ced8f0e740cb7fe73952257ac82b" gracePeriod=30
Sep 06 05:27:31.885150 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:31.885063    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:37.127633 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.127604    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:27:37.240998 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.240906    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-credential-socket\") pod \"86d5a085-cde4-4d06-ab3d-65f550224a3e\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") "
Sep 06 05:27:37.240998 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.240975    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-workload-socket\") pod \"86d5a085-cde4-4d06-ab3d-65f550224a3e\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") "
Sep 06 05:27:37.241262 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.241072    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24s9l\" (UniqueName: \"kubernetes.io/projected/86d5a085-cde4-4d06-ab3d-65f550224a3e-kube-api-access-24s9l\") pod \"86d5a085-cde4-4d06-ab3d-65f550224a3e\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") "
Sep 06 05:27:37.241262 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.241102    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-podinfo\") pod \"86d5a085-cde4-4d06-ab3d-65f550224a3e\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") "
Sep 06 05:27:37.241262 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.241134    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86d5a085-cde4-4d06-ab3d-65f550224a3e-istiod-ca-cert\") pod \"86d5a085-cde4-4d06-ab3d-65f550224a3e\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") "
Sep 06 05:27:37.241262 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.241189    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-workload-certs\") pod \"86d5a085-cde4-4d06-ab3d-65f550224a3e\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") "
Sep 06 05:27:37.241262 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.241230    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-envoy\") pod \"86d5a085-cde4-4d06-ab3d-65f550224a3e\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") "
Sep 06 05:27:37.241262 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.241245    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "86d5a085-cde4-4d06-ab3d-65f550224a3e" (UID: "86d5a085-cde4-4d06-ab3d-65f550224a3e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:37.241633 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.241256    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-token\") pod \"86d5a085-cde4-4d06-ab3d-65f550224a3e\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") "
Sep 06 05:27:37.241633 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.241257    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "86d5a085-cde4-4d06-ab3d-65f550224a3e" (UID: "86d5a085-cde4-4d06-ab3d-65f550224a3e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:37.241633 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.241305    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-data\") pod \"86d5a085-cde4-4d06-ab3d-65f550224a3e\" (UID: \"86d5a085-cde4-4d06-ab3d-65f550224a3e\") "
Sep 06 05:27:37.241796 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.241619    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-data" (OuterVolumeSpecName: "istio-data") pod "86d5a085-cde4-4d06-ab3d-65f550224a3e" (UID: "86d5a085-cde4-4d06-ab3d-65f550224a3e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:37.241796 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.241694    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-credential-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:37.241796 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.241714    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-workload-socket\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:37.241796 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.241768    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86d5a085-cde4-4d06-ab3d-65f550224a3e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "86d5a085-cde4-4d06-ab3d-65f550224a3e" (UID: "86d5a085-cde4-4d06-ab3d-65f550224a3e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:27:37.242007 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.241559    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "86d5a085-cde4-4d06-ab3d-65f550224a3e" (UID: "86d5a085-cde4-4d06-ab3d-65f550224a3e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:37.243361 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.243334    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "86d5a085-cde4-4d06-ab3d-65f550224a3e" (UID: "86d5a085-cde4-4d06-ab3d-65f550224a3e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:27:37.243465 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.243375    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "86d5a085-cde4-4d06-ab3d-65f550224a3e" (UID: "86d5a085-cde4-4d06-ab3d-65f550224a3e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:37.243465 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.243378    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-token" (OuterVolumeSpecName: "istio-token") pod "86d5a085-cde4-4d06-ab3d-65f550224a3e" (UID: "86d5a085-cde4-4d06-ab3d-65f550224a3e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:37.243580 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.243559    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86d5a085-cde4-4d06-ab3d-65f550224a3e-kube-api-access-24s9l" (OuterVolumeSpecName: "kube-api-access-24s9l") pod "86d5a085-cde4-4d06-ab3d-65f550224a3e" (UID: "86d5a085-cde4-4d06-ab3d-65f550224a3e"). InnerVolumeSpecName "kube-api-access-24s9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:37.270756 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.270726    2578 generic.go:356] "Generic (PLEG): container finished" podID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerID="6a0b642357180461692905f2e3bcc14008d3ced8f0e740cb7fe73952257ac82b" exitCode=0
Sep 06 05:27:37.270846 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.270772    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"
Sep 06 05:27:37.270846 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.270779    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq" event={"ID":"86d5a085-cde4-4d06-ab3d-65f550224a3e","Type":"ContainerDied","Data":"6a0b642357180461692905f2e3bcc14008d3ced8f0e740cb7fe73952257ac82b"}
Sep 06 05:27:37.270846 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.270828    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq" event={"ID":"86d5a085-cde4-4d06-ab3d-65f550224a3e","Type":"ContainerDied","Data":"195de068273f779391213050abca544aaf0b5acd8bcfb5202ad6a54553b6bd4c"}
Sep 06 05:27:37.270974 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.270853    2578 scope.go:117] "RemoveContainer" containerID="6a0b642357180461692905f2e3bcc14008d3ced8f0e740cb7fe73952257ac82b"
Sep 06 05:27:37.279598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.279582    2578 scope.go:117] "RemoveContainer" containerID="f51a1464a91f98bc0f9db712a36d9182343e51340b1527584fc339ccd4146f33"
Sep 06 05:27:37.287266 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.287245    2578 scope.go:117] "RemoveContainer" containerID="6a0b642357180461692905f2e3bcc14008d3ced8f0e740cb7fe73952257ac82b"
Sep 06 05:27:37.287516 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:27:37.287499    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a0b642357180461692905f2e3bcc14008d3ced8f0e740cb7fe73952257ac82b\": container with ID starting with 6a0b642357180461692905f2e3bcc14008d3ced8f0e740cb7fe73952257ac82b not found: ID does not exist" containerID="6a0b642357180461692905f2e3bcc14008d3ced8f0e740cb7fe73952257ac82b"
Sep 06 05:27:37.287559 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.287524    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a0b642357180461692905f2e3bcc14008d3ced8f0e740cb7fe73952257ac82b"} err="failed to get container status \"6a0b642357180461692905f2e3bcc14008d3ced8f0e740cb7fe73952257ac82b\": rpc error: code = NotFound desc = could not find container \"6a0b642357180461692905f2e3bcc14008d3ced8f0e740cb7fe73952257ac82b\": container with ID starting with 6a0b642357180461692905f2e3bcc14008d3ced8f0e740cb7fe73952257ac82b not found: ID does not exist"
Sep 06 05:27:37.287559 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.287541    2578 scope.go:117] "RemoveContainer" containerID="f51a1464a91f98bc0f9db712a36d9182343e51340b1527584fc339ccd4146f33"
Sep 06 05:27:37.287811 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:27:37.287792    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f51a1464a91f98bc0f9db712a36d9182343e51340b1527584fc339ccd4146f33\": container with ID starting with f51a1464a91f98bc0f9db712a36d9182343e51340b1527584fc339ccd4146f33 not found: ID does not exist" containerID="f51a1464a91f98bc0f9db712a36d9182343e51340b1527584fc339ccd4146f33"
Sep 06 05:27:37.287858 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.287817    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f51a1464a91f98bc0f9db712a36d9182343e51340b1527584fc339ccd4146f33"} err="failed to get container status \"f51a1464a91f98bc0f9db712a36d9182343e51340b1527584fc339ccd4146f33\": rpc error: code = NotFound desc = could not find container \"f51a1464a91f98bc0f9db712a36d9182343e51340b1527584fc339ccd4146f33\": container with ID starting with f51a1464a91f98bc0f9db712a36d9182343e51340b1527584fc339ccd4146f33 not found: ID does not exist"
Sep 06 05:27:37.293560 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.293521    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"]
Sep 06 05:27:37.295269 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.295248    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-494740-openshift-default-6cbfd69d56-wphzq"]
Sep 06 05:27:37.342326 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.342290    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86d5a085-cde4-4d06-ab3d-65f550224a3e-istiod-ca-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:37.342326 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.342319    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-workload-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:37.342326 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.342327    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-envoy\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:37.342326 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.342335    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-token\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:37.342601 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.342342    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-data\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:37.342601 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.342350    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24s9l\" (UniqueName: \"kubernetes.io/projected/86d5a085-cde4-4d06-ab3d-65f550224a3e-kube-api-access-24s9l\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:37.342601 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:37.342359    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86d5a085-cde4-4d06-ab3d-65f550224a3e-istio-podinfo\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:39.153352 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:39.153320    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86d5a085-cde4-4d06-ab3d-65f550224a3e" path="/var/lib/kubelet/pods/86d5a085-cde4-4d06-ab3d-65f550224a3e/volumes"
Sep 06 05:27:42.604350 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:42.604307    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:42.626138 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:42.626107    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"]
Sep 06 05:27:42.626385 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:42.626359    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5" podUID="e52ec6f2-7df0-488e-b19e-97887e27e36a" containerName="payload-processing" containerID="cri-o://6a44672b91f44362f7ee49259f5fb970344248ce40a8558c698b6c4cc9a2dcf0" gracePeriod=30
Sep 06 05:27:42.704435 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:42.704399    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:42.780799 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:42.780767    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:42.834894 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:42.834862    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:42.894602 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:42.894576    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"
Sep 06 05:27:42.992580 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:42.992534    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52ec6f2-7df0-488e-b19e-97887e27e36a-plugins-config-volume\") pod \"e52ec6f2-7df0-488e-b19e-97887e27e36a\" (UID: \"e52ec6f2-7df0-488e-b19e-97887e27e36a\") "
Sep 06 05:27:42.992771 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:42.992674    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmj6j\" (UniqueName: \"kubernetes.io/projected/e52ec6f2-7df0-488e-b19e-97887e27e36a-kube-api-access-nmj6j\") pod \"e52ec6f2-7df0-488e-b19e-97887e27e36a\" (UID: \"e52ec6f2-7df0-488e-b19e-97887e27e36a\") "
Sep 06 05:27:42.992994 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:42.992957    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e52ec6f2-7df0-488e-b19e-97887e27e36a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e52ec6f2-7df0-488e-b19e-97887e27e36a" (UID: "e52ec6f2-7df0-488e-b19e-97887e27e36a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:27:42.994872 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:42.994845    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e52ec6f2-7df0-488e-b19e-97887e27e36a-kube-api-access-nmj6j" (OuterVolumeSpecName: "kube-api-access-nmj6j") pod "e52ec6f2-7df0-488e-b19e-97887e27e36a" (UID: "e52ec6f2-7df0-488e-b19e-97887e27e36a"). InnerVolumeSpecName "kube-api-access-nmj6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:43.093709 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:43.093662    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52ec6f2-7df0-488e-b19e-97887e27e36a-plugins-config-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:43.093709 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:43.093707    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmj6j\" (UniqueName: \"kubernetes.io/projected/e52ec6f2-7df0-488e-b19e-97887e27e36a-kube-api-access-nmj6j\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:43.149573 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:43.149480    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:43.302727 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:43.302693    2578 generic.go:356] "Generic (PLEG): container finished" podID="e52ec6f2-7df0-488e-b19e-97887e27e36a" containerID="6a44672b91f44362f7ee49259f5fb970344248ce40a8558c698b6c4cc9a2dcf0" exitCode=0
Sep 06 05:27:43.302922 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:43.302736    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5" event={"ID":"e52ec6f2-7df0-488e-b19e-97887e27e36a","Type":"ContainerDied","Data":"6a44672b91f44362f7ee49259f5fb970344248ce40a8558c698b6c4cc9a2dcf0"}
Sep 06 05:27:43.302922 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:43.302792    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5" event={"ID":"e52ec6f2-7df0-488e-b19e-97887e27e36a","Type":"ContainerDied","Data":"8f1accb66ae10d454ca61d7bd36be9a587262926648e1f4e4de24a97050ea499"}
Sep 06 05:27:43.302922 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:43.302746    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"
Sep 06 05:27:43.302922 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:43.302843    2578 scope.go:117] "RemoveContainer" containerID="6a44672b91f44362f7ee49259f5fb970344248ce40a8558c698b6c4cc9a2dcf0"
Sep 06 05:27:43.311912 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:43.311895    2578 scope.go:117] "RemoveContainer" containerID="6a44672b91f44362f7ee49259f5fb970344248ce40a8558c698b6c4cc9a2dcf0"
Sep 06 05:27:43.312225 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:27:43.312196    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a44672b91f44362f7ee49259f5fb970344248ce40a8558c698b6c4cc9a2dcf0\": container with ID starting with 6a44672b91f44362f7ee49259f5fb970344248ce40a8558c698b6c4cc9a2dcf0 not found: ID does not exist" containerID="6a44672b91f44362f7ee49259f5fb970344248ce40a8558c698b6c4cc9a2dcf0"
Sep 06 05:27:43.312306 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:43.312230    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a44672b91f44362f7ee49259f5fb970344248ce40a8558c698b6c4cc9a2dcf0"} err="failed to get container status \"6a44672b91f44362f7ee49259f5fb970344248ce40a8558c698b6c4cc9a2dcf0\": rpc error: code = NotFound desc = could not find container \"6a44672b91f44362f7ee49259f5fb970344248ce40a8558c698b6c4cc9a2dcf0\": container with ID starting with 6a44672b91f44362f7ee49259f5fb970344248ce40a8558c698b6c4cc9a2dcf0 not found: ID does not exist"
Sep 06 05:27:43.317368 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:43.317347    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"]
Sep 06 05:27:43.320841 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:43.320822    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-167b5b-859cff5f8c-5ftc5"]
Sep 06 05:27:45.154181 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:45.154151    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e52ec6f2-7df0-488e-b19e-97887e27e36a" path="/var/lib/kubelet/pods/e52ec6f2-7df0-488e-b19e-97887e27e36a/volumes"
Sep 06 05:27:48.880102 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:48.880073    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:48.998909 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:48.998867    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:49.151421 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:49.151313    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:27:51.623061 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:51.623002    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-q9kvl"]
Sep 06 05:27:51.623513 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:51.623279    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8557bfdbbd-q9kvl" podUID="0c7370b3-6b61-428b-89fc-ebd9162c90da" containerName="manager" containerID="cri-o://fd8bb2125610fae061f9024878c3166861f4721f845cb8e709b831f61a3288d6" gracePeriod=30
Sep 06 05:27:51.870242 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:51.870216    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:27:51.971473 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:51.971382    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stwml\" (UniqueName: \"kubernetes.io/projected/0c7370b3-6b61-428b-89fc-ebd9162c90da-kube-api-access-stwml\") pod \"0c7370b3-6b61-428b-89fc-ebd9162c90da\" (UID: \"0c7370b3-6b61-428b-89fc-ebd9162c90da\") "
Sep 06 05:27:51.971473 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:51.971456    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-metrics-certs\") pod \"0c7370b3-6b61-428b-89fc-ebd9162c90da\" (UID: \"0c7370b3-6b61-428b-89fc-ebd9162c90da\") "
Sep 06 05:27:51.971658 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:51.971558    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-webhook-cert\") pod \"0c7370b3-6b61-428b-89fc-ebd9162c90da\" (UID: \"0c7370b3-6b61-428b-89fc-ebd9162c90da\") "
Sep 06 05:27:51.973495 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:51.973462    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0c7370b3-6b61-428b-89fc-ebd9162c90da" (UID: "0c7370b3-6b61-428b-89fc-ebd9162c90da"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:27:51.973495 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:51.973466    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c7370b3-6b61-428b-89fc-ebd9162c90da-kube-api-access-stwml" (OuterVolumeSpecName: "kube-api-access-stwml") pod "0c7370b3-6b61-428b-89fc-ebd9162c90da" (UID: "0c7370b3-6b61-428b-89fc-ebd9162c90da"). InnerVolumeSpecName "kube-api-access-stwml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:51.973644 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:51.973540    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0c7370b3-6b61-428b-89fc-ebd9162c90da" (UID: "0c7370b3-6b61-428b-89fc-ebd9162c90da"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:27:52.072403 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:52.072365    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-webhook-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:52.072403 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:52.072396    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stwml\" (UniqueName: \"kubernetes.io/projected/0c7370b3-6b61-428b-89fc-ebd9162c90da-kube-api-access-stwml\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:52.072403 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:52.072407    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c7370b3-6b61-428b-89fc-ebd9162c90da-metrics-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:27:52.348520 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:52.348434    2578 generic.go:356] "Generic (PLEG): container finished" podID="0c7370b3-6b61-428b-89fc-ebd9162c90da" containerID="fd8bb2125610fae061f9024878c3166861f4721f845cb8e709b831f61a3288d6" exitCode=0
Sep 06 05:27:52.348520 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:52.348492    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8557bfdbbd-q9kvl" event={"ID":"0c7370b3-6b61-428b-89fc-ebd9162c90da","Type":"ContainerDied","Data":"fd8bb2125610fae061f9024878c3166861f4721f845cb8e709b831f61a3288d6"}
Sep 06 05:27:52.348700 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:52.348541    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8557bfdbbd-q9kvl" event={"ID":"0c7370b3-6b61-428b-89fc-ebd9162c90da","Type":"ContainerDied","Data":"f89569bec6da7086311032f2dbf9ef429bdd85647c3b956dc9d2ecb06ffe9e9e"}
Sep 06 05:27:52.348700 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:52.348546    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8557bfdbbd-q9kvl"
Sep 06 05:27:52.348700 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:52.348565    2578 scope.go:117] "RemoveContainer" containerID="fd8bb2125610fae061f9024878c3166861f4721f845cb8e709b831f61a3288d6"
Sep 06 05:27:52.358482 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:52.358465    2578 scope.go:117] "RemoveContainer" containerID="fd8bb2125610fae061f9024878c3166861f4721f845cb8e709b831f61a3288d6"
Sep 06 05:27:52.358756 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:27:52.358738    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd8bb2125610fae061f9024878c3166861f4721f845cb8e709b831f61a3288d6\": container with ID starting with fd8bb2125610fae061f9024878c3166861f4721f845cb8e709b831f61a3288d6 not found: ID does not exist" containerID="fd8bb2125610fae061f9024878c3166861f4721f845cb8e709b831f61a3288d6"
Sep 06 05:27:52.358805 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:52.358765    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd8bb2125610fae061f9024878c3166861f4721f845cb8e709b831f61a3288d6"} err="failed to get container status \"fd8bb2125610fae061f9024878c3166861f4721f845cb8e709b831f61a3288d6\": rpc error: code = NotFound desc = could not find container \"fd8bb2125610fae061f9024878c3166861f4721f845cb8e709b831f61a3288d6\": container with ID starting with fd8bb2125610fae061f9024878c3166861f4721f845cb8e709b831f61a3288d6 not found: ID does not exist"
Sep 06 05:27:52.369414 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:52.369353    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-q9kvl"]
Sep 06 05:27:52.371384 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:52.371362    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-q9kvl"]
Sep 06 05:27:53.153204 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:27:53.153172    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c7370b3-6b61-428b-89fc-ebd9162c90da" path="/var/lib/kubelet/pods/0c7370b3-6b61-428b-89fc-ebd9162c90da/volumes"
Sep 06 05:28:24.407421 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:24.407382    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:28:26.705659 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:26.705579    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:28:53.135434 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135392    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-w9dcj"]
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135781    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="365b22df-580d-41d2-a343-c861626edcb1" containerName="payload-processing"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135794    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="365b22df-580d-41d2-a343-c861626edcb1" containerName="payload-processing"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135806    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a5e5e07-abda-480c-affd-640fba2c2b95" containerName="payload-processing"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135811    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a5e5e07-abda-480c-affd-640fba2c2b95" containerName="payload-processing"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135822    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerName="istio-proxy"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135827    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerName="istio-proxy"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135837    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerName="istio-proxy"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135842    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerName="istio-proxy"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135850    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c7370b3-6b61-428b-89fc-ebd9162c90da" containerName="manager"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135855    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c7370b3-6b61-428b-89fc-ebd9162c90da" containerName="manager"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135863    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerName="istio-proxy"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135871    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerName="istio-proxy"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135882    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1f7bf25-c781-476a-9a9a-8a1c95358441" containerName="maas-api"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135887    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1f7bf25-c781-476a-9a9a-8a1c95358441" containerName="maas-api"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135896    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a65b31e0-ed0c-4e66-913a-1f38b47c37db" containerName="payload-processing"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135902    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65b31e0-ed0c-4e66-913a-1f38b47c37db" containerName="payload-processing"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135911    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e52ec6f2-7df0-488e-b19e-97887e27e36a" containerName="payload-processing"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135916    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e52ec6f2-7df0-488e-b19e-97887e27e36a" containerName="payload-processing"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135982    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="365b22df-580d-41d2-a343-c861626edcb1" containerName="payload-processing"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135991    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e52ec6f2-7df0-488e-b19e-97887e27e36a" containerName="payload-processing"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.135997    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1f7bf25-c781-476a-9a9a-8a1c95358441" containerName="maas-api"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.136003    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerName="istio-proxy"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.136010    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerName="istio-proxy"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.136029    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c7370b3-6b61-428b-89fc-ebd9162c90da" containerName="manager"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.136038    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a65b31e0-ed0c-4e66-913a-1f38b47c37db" containerName="payload-processing"
Sep 06 05:28:53.137811 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.136046    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a5e5e07-abda-480c-affd-640fba2c2b95" containerName="payload-processing"
Sep 06 05:28:53.138917 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.138900    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:28:53.141347 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.141324    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 06 05:28:53.142565 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.142542    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vpcfl\""
Sep 06 05:28:53.142713 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.142542    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 06 05:28:53.142713 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.142554    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 06 05:28:53.155109 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.155083    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-w9dcj"]
Sep 06 05:28:53.236668 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.236634    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b92923-4f06-4f18-8828-5f8d04dbea1c-metrics-certs\") pod \"maas-controller-8557bfdbbd-w9dcj\" (UID: \"41b92923-4f06-4f18-8828-5f8d04dbea1c\") " pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:28:53.236872 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.236689    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41b92923-4f06-4f18-8828-5f8d04dbea1c-webhook-cert\") pod \"maas-controller-8557bfdbbd-w9dcj\" (UID: \"41b92923-4f06-4f18-8828-5f8d04dbea1c\") " pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:28:53.236872 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.236705    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l59x8\" (UniqueName: \"kubernetes.io/projected/41b92923-4f06-4f18-8828-5f8d04dbea1c-kube-api-access-l59x8\") pod \"maas-controller-8557bfdbbd-w9dcj\" (UID: \"41b92923-4f06-4f18-8828-5f8d04dbea1c\") " pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:28:53.338101 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.338052    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b92923-4f06-4f18-8828-5f8d04dbea1c-metrics-certs\") pod \"maas-controller-8557bfdbbd-w9dcj\" (UID: \"41b92923-4f06-4f18-8828-5f8d04dbea1c\") " pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:28:53.338306 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.338113    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41b92923-4f06-4f18-8828-5f8d04dbea1c-webhook-cert\") pod \"maas-controller-8557bfdbbd-w9dcj\" (UID: \"41b92923-4f06-4f18-8828-5f8d04dbea1c\") " pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:28:53.338306 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.338139    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l59x8\" (UniqueName: \"kubernetes.io/projected/41b92923-4f06-4f18-8828-5f8d04dbea1c-kube-api-access-l59x8\") pod \"maas-controller-8557bfdbbd-w9dcj\" (UID: \"41b92923-4f06-4f18-8828-5f8d04dbea1c\") " pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:28:53.340643 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.340614    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41b92923-4f06-4f18-8828-5f8d04dbea1c-webhook-cert\") pod \"maas-controller-8557bfdbbd-w9dcj\" (UID: \"41b92923-4f06-4f18-8828-5f8d04dbea1c\") " pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:28:53.340643 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.340625    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b92923-4f06-4f18-8828-5f8d04dbea1c-metrics-certs\") pod \"maas-controller-8557bfdbbd-w9dcj\" (UID: \"41b92923-4f06-4f18-8828-5f8d04dbea1c\") " pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:28:53.345333 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.345306    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l59x8\" (UniqueName: \"kubernetes.io/projected/41b92923-4f06-4f18-8828-5f8d04dbea1c-kube-api-access-l59x8\") pod \"maas-controller-8557bfdbbd-w9dcj\" (UID: \"41b92923-4f06-4f18-8828-5f8d04dbea1c\") " pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:28:53.450558 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.450468    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:28:53.719827 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.719800    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-w9dcj"]
Sep 06 05:28:53.721729 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:28:53.721701    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41b92923_4f06_4f18_8828_5f8d04dbea1c.slice/crio-a852fda6efdad659366a8f52fa078e55f0dadaf080ec660d8762b3a3290d1fda WatchSource:0}: Error finding container a852fda6efdad659366a8f52fa078e55f0dadaf080ec660d8762b3a3290d1fda: Status 404 returned error can't find the container with id a852fda6efdad659366a8f52fa078e55f0dadaf080ec660d8762b3a3290d1fda
Sep 06 05:28:53.723167 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:53.723149    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 05:28:54.653951 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:54.653914    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8557bfdbbd-w9dcj" event={"ID":"41b92923-4f06-4f18-8828-5f8d04dbea1c","Type":"ContainerStarted","Data":"fc55afa454b5d6824fc636d7c134ee8c4bfc92fc5e61d030a67d4d00c2e2e8e1"}
Sep 06 05:28:54.654345 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:54.653977    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8557bfdbbd-w9dcj" event={"ID":"41b92923-4f06-4f18-8828-5f8d04dbea1c","Type":"ContainerStarted","Data":"a852fda6efdad659366a8f52fa078e55f0dadaf080ec660d8762b3a3290d1fda"}
Sep 06 05:28:54.654345 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:54.654125    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:28:54.670972 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:28:54.670918    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8557bfdbbd-w9dcj" podStartSLOduration=1.120761617 podStartE2EDuration="1.67090258s" totalImagesPullingTime="550.140963ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:28:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:28:54.669216757 +0000 UTC m=+2002.038672003" watchObservedRunningTime="2026-09-06 05:28:54.67090258 +0000 UTC m=+2002.040357826"
Sep 06 05:29:05.661909 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:05.661877    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:29:13.692502 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:13.692467    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:29:14.903835 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:14.903797    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:29:15.009792 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.009754    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-w9dcj"]
Sep 06 05:29:15.010027 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.009993    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8557bfdbbd-w9dcj" podUID="41b92923-4f06-4f18-8828-5f8d04dbea1c" containerName="manager" containerID="cri-o://fc55afa454b5d6824fc636d7c134ee8c4bfc92fc5e61d030a67d4d00c2e2e8e1" gracePeriod=30
Sep 06 05:29:15.260104 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.260075    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:29:15.343091 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.343058    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b92923-4f06-4f18-8828-5f8d04dbea1c-metrics-certs\") pod \"41b92923-4f06-4f18-8828-5f8d04dbea1c\" (UID: \"41b92923-4f06-4f18-8828-5f8d04dbea1c\") "
Sep 06 05:29:15.343253 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.343168    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l59x8\" (UniqueName: \"kubernetes.io/projected/41b92923-4f06-4f18-8828-5f8d04dbea1c-kube-api-access-l59x8\") pod \"41b92923-4f06-4f18-8828-5f8d04dbea1c\" (UID: \"41b92923-4f06-4f18-8828-5f8d04dbea1c\") "
Sep 06 05:29:15.343253 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.343222    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41b92923-4f06-4f18-8828-5f8d04dbea1c-webhook-cert\") pod \"41b92923-4f06-4f18-8828-5f8d04dbea1c\" (UID: \"41b92923-4f06-4f18-8828-5f8d04dbea1c\") "
Sep 06 05:29:15.345236 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.345203    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41b92923-4f06-4f18-8828-5f8d04dbea1c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "41b92923-4f06-4f18-8828-5f8d04dbea1c" (UID: "41b92923-4f06-4f18-8828-5f8d04dbea1c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:29:15.345354 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.345273    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41b92923-4f06-4f18-8828-5f8d04dbea1c-kube-api-access-l59x8" (OuterVolumeSpecName: "kube-api-access-l59x8") pod "41b92923-4f06-4f18-8828-5f8d04dbea1c" (UID: "41b92923-4f06-4f18-8828-5f8d04dbea1c"). InnerVolumeSpecName "kube-api-access-l59x8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:29:15.345563 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.345544    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41b92923-4f06-4f18-8828-5f8d04dbea1c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "41b92923-4f06-4f18-8828-5f8d04dbea1c" (UID: "41b92923-4f06-4f18-8828-5f8d04dbea1c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:29:15.444491 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.444378    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l59x8\" (UniqueName: \"kubernetes.io/projected/41b92923-4f06-4f18-8828-5f8d04dbea1c-kube-api-access-l59x8\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:29:15.444491 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.444427    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41b92923-4f06-4f18-8828-5f8d04dbea1c-webhook-cert\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:29:15.444491 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.444443    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b92923-4f06-4f18-8828-5f8d04dbea1c-metrics-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:29:15.763869 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.763781    2578 generic.go:356] "Generic (PLEG): container finished" podID="41b92923-4f06-4f18-8828-5f8d04dbea1c" containerID="fc55afa454b5d6824fc636d7c134ee8c4bfc92fc5e61d030a67d4d00c2e2e8e1" exitCode=0
Sep 06 05:29:15.763869 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.763816    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8557bfdbbd-w9dcj" event={"ID":"41b92923-4f06-4f18-8828-5f8d04dbea1c","Type":"ContainerDied","Data":"fc55afa454b5d6824fc636d7c134ee8c4bfc92fc5e61d030a67d4d00c2e2e8e1"}
Sep 06 05:29:15.764151 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.763883    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8557bfdbbd-w9dcj" event={"ID":"41b92923-4f06-4f18-8828-5f8d04dbea1c","Type":"ContainerDied","Data":"a852fda6efdad659366a8f52fa078e55f0dadaf080ec660d8762b3a3290d1fda"}
Sep 06 05:29:15.764151 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.763830    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8557bfdbbd-w9dcj"
Sep 06 05:29:15.764151 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.763913    2578 scope.go:117] "RemoveContainer" containerID="fc55afa454b5d6824fc636d7c134ee8c4bfc92fc5e61d030a67d4d00c2e2e8e1"
Sep 06 05:29:15.773987 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.773964    2578 scope.go:117] "RemoveContainer" containerID="fc55afa454b5d6824fc636d7c134ee8c4bfc92fc5e61d030a67d4d00c2e2e8e1"
Sep 06 05:29:15.774297 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:29:15.774277    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc55afa454b5d6824fc636d7c134ee8c4bfc92fc5e61d030a67d4d00c2e2e8e1\": container with ID starting with fc55afa454b5d6824fc636d7c134ee8c4bfc92fc5e61d030a67d4d00c2e2e8e1 not found: ID does not exist" containerID="fc55afa454b5d6824fc636d7c134ee8c4bfc92fc5e61d030a67d4d00c2e2e8e1"
Sep 06 05:29:15.774369 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.774304    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc55afa454b5d6824fc636d7c134ee8c4bfc92fc5e61d030a67d4d00c2e2e8e1"} err="failed to get container status \"fc55afa454b5d6824fc636d7c134ee8c4bfc92fc5e61d030a67d4d00c2e2e8e1\": rpc error: code = NotFound desc = could not find container \"fc55afa454b5d6824fc636d7c134ee8c4bfc92fc5e61d030a67d4d00c2e2e8e1\": container with ID starting with fc55afa454b5d6824fc636d7c134ee8c4bfc92fc5e61d030a67d4d00c2e2e8e1 not found: ID does not exist"
Sep 06 05:29:15.784800 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.784773    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-w9dcj"]
Sep 06 05:29:15.786362 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:15.786336    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-w9dcj"]
Sep 06 05:29:16.442752 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.442721    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:29:16.562640 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.562604    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-qzs94"]
Sep 06 05:29:16.562969 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.562957    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41b92923-4f06-4f18-8828-5f8d04dbea1c" containerName="manager"
Sep 06 05:29:16.563060 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.562971    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="41b92923-4f06-4f18-8828-5f8d04dbea1c" containerName="manager"
Sep 06 05:29:16.563060 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.562981    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerName="istio-proxy"
Sep 06 05:29:16.563060 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.562986    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerName="istio-proxy"
Sep 06 05:29:16.563182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.563074    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="86d5a085-cde4-4d06-ab3d-65f550224a3e" containerName="istio-proxy"
Sep 06 05:29:16.563182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.563083    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="38ef5202-290f-4b76-b7b2-4f1e3fbab725" containerName="istio-proxy"
Sep 06 05:29:16.563182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.563090    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="41b92923-4f06-4f18-8828-5f8d04dbea1c" containerName="manager"
Sep 06 05:29:16.567415 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.567392    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8557bfdbbd-qzs94"
Sep 06 05:29:16.570558 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.570539    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vpcfl\""
Sep 06 05:29:16.570671 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.570544    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 06 05:29:16.570859 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.570841    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 06 05:29:16.570983 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.570870    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 06 05:29:16.575846 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.575822    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-qzs94"]
Sep 06 05:29:16.656228 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.656191    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnjb8\" (UniqueName: \"kubernetes.io/projected/9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3-kube-api-access-vnjb8\") pod \"maas-controller-8557bfdbbd-qzs94\" (UID: \"9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3\") " pod="opendatahub/maas-controller-8557bfdbbd-qzs94"
Sep 06 05:29:16.656407 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.656237    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3-webhook-cert\") pod \"maas-controller-8557bfdbbd-qzs94\" (UID: \"9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3\") " pod="opendatahub/maas-controller-8557bfdbbd-qzs94"
Sep 06 05:29:16.656407 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.656369    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3-metrics-certs\") pod \"maas-controller-8557bfdbbd-qzs94\" (UID: \"9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3\") " pod="opendatahub/maas-controller-8557bfdbbd-qzs94"
Sep 06 05:29:16.757602 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.757501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3-metrics-certs\") pod \"maas-controller-8557bfdbbd-qzs94\" (UID: \"9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3\") " pod="opendatahub/maas-controller-8557bfdbbd-qzs94"
Sep 06 05:29:16.757602 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.757548    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjb8\" (UniqueName: \"kubernetes.io/projected/9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3-kube-api-access-vnjb8\") pod \"maas-controller-8557bfdbbd-qzs94\" (UID: \"9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3\") " pod="opendatahub/maas-controller-8557bfdbbd-qzs94"
Sep 06 05:29:16.757602 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.757566    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3-webhook-cert\") pod \"maas-controller-8557bfdbbd-qzs94\" (UID: \"9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3\") " pod="opendatahub/maas-controller-8557bfdbbd-qzs94"
Sep 06 05:29:16.760095 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.760070    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3-metrics-certs\") pod \"maas-controller-8557bfdbbd-qzs94\" (UID: \"9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3\") " pod="opendatahub/maas-controller-8557bfdbbd-qzs94"
Sep 06 05:29:16.760206 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.760104    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3-webhook-cert\") pod \"maas-controller-8557bfdbbd-qzs94\" (UID: \"9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3\") " pod="opendatahub/maas-controller-8557bfdbbd-qzs94"
Sep 06 05:29:16.765483 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.765458    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnjb8\" (UniqueName: \"kubernetes.io/projected/9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3-kube-api-access-vnjb8\") pod \"maas-controller-8557bfdbbd-qzs94\" (UID: \"9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3\") " pod="opendatahub/maas-controller-8557bfdbbd-qzs94"
Sep 06 05:29:16.878446 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:16.878414    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8557bfdbbd-qzs94"
Sep 06 05:29:17.156226 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:29:17.156196    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca03c2c_e39e_4086_b06c_f6a5bbd04ec3.slice/crio-aa7f5de88e52247965e259a06a585dae4484da1529640094f8bfbc77ab06e167 WatchSource:0}: Error finding container aa7f5de88e52247965e259a06a585dae4484da1529640094f8bfbc77ab06e167: Status 404 returned error can't find the container with id aa7f5de88e52247965e259a06a585dae4484da1529640094f8bfbc77ab06e167
Sep 06 05:29:17.156226 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:17.156218    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41b92923-4f06-4f18-8828-5f8d04dbea1c" path="/var/lib/kubelet/pods/41b92923-4f06-4f18-8828-5f8d04dbea1c/volumes"
Sep 06 05:29:17.156489 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:17.156472    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8557bfdbbd-qzs94"]
Sep 06 05:29:17.777280 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:17.777192    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8557bfdbbd-qzs94" event={"ID":"9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3","Type":"ContainerStarted","Data":"9f4040a2b44aad4c70e7235bc8021bde715eb7be32524e4b8faf77e32ea5307d"}
Sep 06 05:29:17.777627 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:17.777282    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8557bfdbbd-qzs94" event={"ID":"9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3","Type":"ContainerStarted","Data":"aa7f5de88e52247965e259a06a585dae4484da1529640094f8bfbc77ab06e167"}
Sep 06 05:29:17.777627 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:17.777330    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8557bfdbbd-qzs94"
Sep 06 05:29:17.793937 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:17.793876    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8557bfdbbd-qzs94" podStartSLOduration=1.49349362 podStartE2EDuration="1.793855735s" totalImagesPullingTime="300.362115ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:29:17.791377199 +0000 UTC m=+2025.160832448" watchObservedRunningTime="2026-09-06 05:29:17.793855735 +0000 UTC m=+2025.163310982"
Sep 06 05:29:28.785096 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:29:28.785064    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8557bfdbbd-qzs94"
Sep 06 05:30:25.124784 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:25.124740    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"]
Sep 06 05:30:25.125402 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:25.125003    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h" podUID="b1aaff57-1941-48e7-b0e1-435701602c80" containerName="manager" containerID="cri-o://15582671f95fdff0d75a4bea3d735a9f76f23dc1314ec3deb718ca3befbced1a" gracePeriod=10
Sep 06 05:30:26.070920 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.070896    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"
Sep 06 05:30:26.122289 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.122254    2578 generic.go:356] "Generic (PLEG): container finished" podID="b1aaff57-1941-48e7-b0e1-435701602c80" containerID="15582671f95fdff0d75a4bea3d735a9f76f23dc1314ec3deb718ca3befbced1a" exitCode=0
Sep 06 05:30:26.122468 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.122307    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"
Sep 06 05:30:26.122468 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.122355    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h" event={"ID":"b1aaff57-1941-48e7-b0e1-435701602c80","Type":"ContainerDied","Data":"15582671f95fdff0d75a4bea3d735a9f76f23dc1314ec3deb718ca3befbced1a"}
Sep 06 05:30:26.122468 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.122421    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h" event={"ID":"b1aaff57-1941-48e7-b0e1-435701602c80","Type":"ContainerDied","Data":"b3dcefe3c563a32131547c448048a1a7d687f5f8e8c844a19ee391a866006782"}
Sep 06 05:30:26.122468 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.122450    2578 scope.go:117] "RemoveContainer" containerID="15582671f95fdff0d75a4bea3d735a9f76f23dc1314ec3deb718ca3befbced1a"
Sep 06 05:30:26.133739 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.133573    2578 scope.go:117] "RemoveContainer" containerID="15582671f95fdff0d75a4bea3d735a9f76f23dc1314ec3deb718ca3befbced1a"
Sep 06 05:30:26.133998 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:30:26.133900    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15582671f95fdff0d75a4bea3d735a9f76f23dc1314ec3deb718ca3befbced1a\": container with ID starting with 15582671f95fdff0d75a4bea3d735a9f76f23dc1314ec3deb718ca3befbced1a not found: ID does not exist" containerID="15582671f95fdff0d75a4bea3d735a9f76f23dc1314ec3deb718ca3befbced1a"
Sep 06 05:30:26.133998 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.133925    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15582671f95fdff0d75a4bea3d735a9f76f23dc1314ec3deb718ca3befbced1a"} err="failed to get container status \"15582671f95fdff0d75a4bea3d735a9f76f23dc1314ec3deb718ca3befbced1a\": rpc error: code = NotFound desc = could not find container \"15582671f95fdff0d75a4bea3d735a9f76f23dc1314ec3deb718ca3befbced1a\": container with ID starting with 15582671f95fdff0d75a4bea3d735a9f76f23dc1314ec3deb718ca3befbced1a not found: ID does not exist"
Sep 06 05:30:26.206472 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.206389    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krwb2\" (UniqueName: \"kubernetes.io/projected/b1aaff57-1941-48e7-b0e1-435701602c80-kube-api-access-krwb2\") pod \"b1aaff57-1941-48e7-b0e1-435701602c80\" (UID: \"b1aaff57-1941-48e7-b0e1-435701602c80\") "
Sep 06 05:30:26.206628 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.206516    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1aaff57-1941-48e7-b0e1-435701602c80-extensions-socket-volume\") pod \"b1aaff57-1941-48e7-b0e1-435701602c80\" (UID: \"b1aaff57-1941-48e7-b0e1-435701602c80\") "
Sep 06 05:30:26.206854 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.206832    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1aaff57-1941-48e7-b0e1-435701602c80-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b1aaff57-1941-48e7-b0e1-435701602c80" (UID: "b1aaff57-1941-48e7-b0e1-435701602c80"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:30:26.208479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.208451    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1aaff57-1941-48e7-b0e1-435701602c80-kube-api-access-krwb2" (OuterVolumeSpecName: "kube-api-access-krwb2") pod "b1aaff57-1941-48e7-b0e1-435701602c80" (UID: "b1aaff57-1941-48e7-b0e1-435701602c80"). InnerVolumeSpecName "kube-api-access-krwb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:30:26.307512 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.307473    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1aaff57-1941-48e7-b0e1-435701602c80-extensions-socket-volume\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:30:26.307512 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.307509    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krwb2\" (UniqueName: \"kubernetes.io/projected/b1aaff57-1941-48e7-b0e1-435701602c80-kube-api-access-krwb2\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:30:26.442990 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.442962    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"]
Sep 06 05:30:26.446249 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:26.446225    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-bls2h"]
Sep 06 05:30:27.152466 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:30:27.152437    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1aaff57-1941-48e7-b0e1-435701602c80" path="/var/lib/kubelet/pods/b1aaff57-1941-48e7-b0e1-435701602c80/volumes"
Sep 06 05:31:25.812101 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:25.812004    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr"]
Sep 06 05:31:25.812691 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:25.812397    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1aaff57-1941-48e7-b0e1-435701602c80" containerName="manager"
Sep 06 05:31:25.812691 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:25.812408    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1aaff57-1941-48e7-b0e1-435701602c80" containerName="manager"
Sep 06 05:31:25.812691 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:25.812463    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1aaff57-1941-48e7-b0e1-435701602c80" containerName="manager"
Sep 06 05:31:25.815406 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:25.815388    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr"
Sep 06 05:31:25.817667 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:25.817642    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bs2v2\""
Sep 06 05:31:25.825780 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:25.825758    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr"]
Sep 06 05:31:25.843114 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:25.843066    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/368363d8-2c06-4076-98b0-dcb96679ddf9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-77kqr\" (UID: \"368363d8-2c06-4076-98b0-dcb96679ddf9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr"
Sep 06 05:31:25.843244 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:25.843133    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vstck\" (UniqueName: \"kubernetes.io/projected/368363d8-2c06-4076-98b0-dcb96679ddf9-kube-api-access-vstck\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-77kqr\" (UID: \"368363d8-2c06-4076-98b0-dcb96679ddf9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr"
Sep 06 05:31:25.944150 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:25.944116    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/368363d8-2c06-4076-98b0-dcb96679ddf9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-77kqr\" (UID: \"368363d8-2c06-4076-98b0-dcb96679ddf9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr"
Sep 06 05:31:25.944322 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:25.944171    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vstck\" (UniqueName: \"kubernetes.io/projected/368363d8-2c06-4076-98b0-dcb96679ddf9-kube-api-access-vstck\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-77kqr\" (UID: \"368363d8-2c06-4076-98b0-dcb96679ddf9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr"
Sep 06 05:31:25.944522 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:25.944484    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/368363d8-2c06-4076-98b0-dcb96679ddf9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-77kqr\" (UID: \"368363d8-2c06-4076-98b0-dcb96679ddf9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr"
Sep 06 05:31:25.952081 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:25.952031    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vstck\" (UniqueName: \"kubernetes.io/projected/368363d8-2c06-4076-98b0-dcb96679ddf9-kube-api-access-vstck\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-77kqr\" (UID: \"368363d8-2c06-4076-98b0-dcb96679ddf9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr"
Sep 06 05:31:26.126636 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:26.126603    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr"
Sep 06 05:31:26.400082 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:26.400055    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr"]
Sep 06 05:31:26.402856 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:31:26.402825    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod368363d8_2c06_4076_98b0_dcb96679ddf9.slice/crio-8e3f25d562c37c8e75ddafeecc505190812da492bc28802fcff5827bd64bf431 WatchSource:0}: Error finding container 8e3f25d562c37c8e75ddafeecc505190812da492bc28802fcff5827bd64bf431: Status 404 returned error can't find the container with id 8e3f25d562c37c8e75ddafeecc505190812da492bc28802fcff5827bd64bf431
Sep 06 05:31:27.410283 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:27.410232    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr" event={"ID":"368363d8-2c06-4076-98b0-dcb96679ddf9","Type":"ContainerStarted","Data":"aa3ce742e8546871a7ff6da19dd1854ca1003082331d73035d88144c42febfa2"}
Sep 06 05:31:27.410775 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:27.410328    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr" event={"ID":"368363d8-2c06-4076-98b0-dcb96679ddf9","Type":"ContainerStarted","Data":"8e3f25d562c37c8e75ddafeecc505190812da492bc28802fcff5827bd64bf431"}
Sep 06 05:31:27.410775 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:27.410363    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr"
Sep 06 05:31:27.428828 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:27.428775    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr" podStartSLOduration=2.428757036 podStartE2EDuration="2.428757036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:31:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:31:27.427042114 +0000 UTC m=+2154.796497360" watchObservedRunningTime="2026-09-06 05:31:27.428757036 +0000 UTC m=+2154.798212283"
Sep 06 05:31:38.417100 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:31:38.417061    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-77kqr"
Sep 06 05:32:21.741180 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.741147    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"]
Sep 06 05:32:21.744654 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.744636    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.746693 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.746674    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-5ec92654-kserve-self-signed-certs\""
Sep 06 05:32:21.751788 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.751759    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"]
Sep 06 05:32:21.801192 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.801157    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:32:21.852394 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.852360    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-tls-certs\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.852394 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.852396    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hx7q\" (UniqueName: \"kubernetes.io/projected/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-kube-api-access-6hx7q\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.852605 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.852417    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-dshm\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.852605 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.852438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-model-cache\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.852605 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.852478    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-home\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.852605 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.852500    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-tmp-dir\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.953734 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.953628    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-dshm\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.953892 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.953764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-model-cache\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.953892 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.953811    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-home\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.953892 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.953842    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-tmp-dir\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.954069 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.953926    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-tls-certs\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.954069 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.953952    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hx7q\" (UniqueName: \"kubernetes.io/projected/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-kube-api-access-6hx7q\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.954200 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.954174    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-model-cache\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.954200 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.954188    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-home\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.954330 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.954238    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-tmp-dir\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.955930 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.955909    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-dshm\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.956407 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.956385    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-tls-certs\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:21.961270 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:21.961251    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hx7q\" (UniqueName: \"kubernetes.io/projected/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-kube-api-access-6hx7q\") pod \"e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") " pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:22.054835 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:22.054738    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:22.328082 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:22.328033    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"]
Sep 06 05:32:22.330538 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:32:22.330507    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40451dfc_2530_4ca8_a53f_bcb99a73b0f5.slice/crio-31c4d1a1961ba5a9d5de13d5d87ee740ddb9158dceb710d5f18b4eb230a8c92d WatchSource:0}: Error finding container 31c4d1a1961ba5a9d5de13d5d87ee740ddb9158dceb710d5f18b4eb230a8c92d: Status 404 returned error can't find the container with id 31c4d1a1961ba5a9d5de13d5d87ee740ddb9158dceb710d5f18b4eb230a8c92d
Sep 06 05:32:22.686933 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:22.686892    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2" event={"ID":"40451dfc-2530-4ca8-a53f-bcb99a73b0f5","Type":"ContainerStarted","Data":"de8c8dda54ab18ffc98523016fdd6c2efe839f914ea5d6bde5a6efd92598e74f"}
Sep 06 05:32:22.686933 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:22.686933    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2" event={"ID":"40451dfc-2530-4ca8-a53f-bcb99a73b0f5","Type":"ContainerStarted","Data":"31c4d1a1961ba5a9d5de13d5d87ee740ddb9158dceb710d5f18b4eb230a8c92d"}
Sep 06 05:32:22.703227 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:22.703182    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2" podStartSLOduration=1.703166647 podStartE2EDuration="1.703166647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:32:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:32:22.701711723 +0000 UTC m=+2210.071166970" watchObservedRunningTime="2026-09-06 05:32:22.703166647 +0000 UTC m=+2210.072621922"
Sep 06 05:32:24.444399 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.444359    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"]
Sep 06 05:32:24.447908 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.447884    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.450291 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.450268    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-5ec92654-kserve-self-signed-certs\""
Sep 06 05:32:24.455195 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.455171    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"]
Sep 06 05:32:24.473497 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.473470    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:32:24.578215 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.578170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55614266-7c57-4339-9a97-77abfa06b43e-tls-certs\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.578385 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.578220    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvxj5\" (UniqueName: \"kubernetes.io/projected/55614266-7c57-4339-9a97-77abfa06b43e-kube-api-access-zvxj5\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.578385 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.578301    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-model-cache\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.578475 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.578425    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-dshm\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.578475 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.578446    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-home\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.578573 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.578486    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-tmp-dir\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.679872 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.679834    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-dshm\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.679872 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.679871    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-home\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.680161 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.679901    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-tmp-dir\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.680161 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.679929    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55614266-7c57-4339-9a97-77abfa06b43e-tls-certs\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.680161 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.679948    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvxj5\" (UniqueName: \"kubernetes.io/projected/55614266-7c57-4339-9a97-77abfa06b43e-kube-api-access-zvxj5\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.680161 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.679974    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-model-cache\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.680375 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.680301    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-tmp-dir\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.680375 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.680344    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-home\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.680459 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.680403    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-model-cache\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.682249 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.682228    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-dshm\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.682540 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.682521    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55614266-7c57-4339-9a97-77abfa06b43e-tls-certs\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.687632 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.687607    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvxj5\" (UniqueName: \"kubernetes.io/projected/55614266-7c57-4339-9a97-77abfa06b43e-kube-api-access-zvxj5\") pod \"e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") " pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:24.760907 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:24.760811    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:25.033791 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:25.033761    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"]
Sep 06 05:32:25.035734 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:32:25.035701    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55614266_7c57_4339_9a97_77abfa06b43e.slice/crio-0cd5d122d81b725840a87ba378b68da440bc8e6f6068c04a6a491f33ab508bd6 WatchSource:0}: Error finding container 0cd5d122d81b725840a87ba378b68da440bc8e6f6068c04a6a491f33ab508bd6: Status 404 returned error can't find the container with id 0cd5d122d81b725840a87ba378b68da440bc8e6f6068c04a6a491f33ab508bd6
Sep 06 05:32:25.709922 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:25.709875    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r" event={"ID":"55614266-7c57-4339-9a97-77abfa06b43e","Type":"ContainerStarted","Data":"a23f4f19956634875a3f213487c194d09f5f6368ff9e905ac2eeeca2a192227d"}
Sep 06 05:32:25.710444 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:25.709933    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r" event={"ID":"55614266-7c57-4339-9a97-77abfa06b43e","Type":"ContainerStarted","Data":"0cd5d122d81b725840a87ba378b68da440bc8e6f6068c04a6a491f33ab508bd6"}
Sep 06 05:32:25.726702 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:25.726640    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r" podStartSLOduration=1.7266185809999999 podStartE2EDuration="1.726618581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:32:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:32:25.724051518 +0000 UTC m=+2213.093506773" watchObservedRunningTime="2026-09-06 05:32:25.726618581 +0000 UTC m=+2213.096073830"
Sep 06 05:32:32.055449 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:32.055415    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:32.055833 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:32.055465    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:32.068154 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:32.068128    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:32.758041 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:32.757989    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:34.761482 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:34.761445    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:34.762030 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:34.761499    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:34.773985 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:34.773963    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:35.771474 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:35.771442    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:38.618999 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:38.618959    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"]
Sep 06 05:32:38.632390 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:38.632241    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:32:40.406291 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:40.406240    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"]
Sep 06 05:32:40.425177 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:40.425143    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:32:41.582085 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:41.582049    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:32:41.632521 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:41.632477    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:32:53.646091 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.646004    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2" podUID="40451dfc-2530-4ca8-a53f-bcb99a73b0f5" containerName="main" containerID="cri-o://de8c8dda54ab18ffc98523016fdd6c2efe839f914ea5d6bde5a6efd92598e74f" gracePeriod=45
Sep 06 05:32:53.856173 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.856140    2578 generic.go:356] "Generic (PLEG): container finished" podID="40451dfc-2530-4ca8-a53f-bcb99a73b0f5" containerID="de8c8dda54ab18ffc98523016fdd6c2efe839f914ea5d6bde5a6efd92598e74f" exitCode=0
Sep 06 05:32:53.856329 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.856216    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2" event={"ID":"40451dfc-2530-4ca8-a53f-bcb99a73b0f5","Type":"ContainerDied","Data":"de8c8dda54ab18ffc98523016fdd6c2efe839f914ea5d6bde5a6efd92598e74f"}
Sep 06 05:32:53.887727 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.887703    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:53.962313 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.962215    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-tls-certs\") pod \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") "
Sep 06 05:32:53.962313 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.962279    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-model-cache\") pod \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") "
Sep 06 05:32:53.962567 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.962341    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-tmp-dir\") pod \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") "
Sep 06 05:32:53.962567 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.962364    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-home\") pod \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") "
Sep 06 05:32:53.962567 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.962427    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-dshm\") pod \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") "
Sep 06 05:32:53.962567 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.962468    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hx7q\" (UniqueName: \"kubernetes.io/projected/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-kube-api-access-6hx7q\") pod \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\" (UID: \"40451dfc-2530-4ca8-a53f-bcb99a73b0f5\") "
Sep 06 05:32:53.962757 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.962576    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-model-cache" (OuterVolumeSpecName: "model-cache") pod "40451dfc-2530-4ca8-a53f-bcb99a73b0f5" (UID: "40451dfc-2530-4ca8-a53f-bcb99a73b0f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:32:53.962757 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.962663    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "40451dfc-2530-4ca8-a53f-bcb99a73b0f5" (UID: "40451dfc-2530-4ca8-a53f-bcb99a73b0f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:32:53.962757 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.962712    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-home" (OuterVolumeSpecName: "home") pod "40451dfc-2530-4ca8-a53f-bcb99a73b0f5" (UID: "40451dfc-2530-4ca8-a53f-bcb99a73b0f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:32:53.962914 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.962859    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-tmp-dir\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:32:53.962914 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.962874    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-home\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:32:53.962914 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.962883    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-model-cache\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:32:53.964440 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.964413    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-dshm" (OuterVolumeSpecName: "dshm") pod "40451dfc-2530-4ca8-a53f-bcb99a73b0f5" (UID: "40451dfc-2530-4ca8-a53f-bcb99a73b0f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:32:53.964543 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.964492    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "40451dfc-2530-4ca8-a53f-bcb99a73b0f5" (UID: "40451dfc-2530-4ca8-a53f-bcb99a73b0f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:32:53.964543 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:53.964508    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-kube-api-access-6hx7q" (OuterVolumeSpecName: "kube-api-access-6hx7q") pod "40451dfc-2530-4ca8-a53f-bcb99a73b0f5" (UID: "40451dfc-2530-4ca8-a53f-bcb99a73b0f5"). InnerVolumeSpecName "kube-api-access-6hx7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:32:54.064245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:54.064211    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-dshm\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:32:54.064245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:54.064240    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hx7q\" (UniqueName: \"kubernetes.io/projected/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-kube-api-access-6hx7q\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:32:54.064245 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:54.064250    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40451dfc-2530-4ca8-a53f-bcb99a73b0f5-tls-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:32:54.863396 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:54.863369    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"
Sep 06 05:32:54.863836 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:54.863419    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2" event={"ID":"40451dfc-2530-4ca8-a53f-bcb99a73b0f5","Type":"ContainerDied","Data":"31c4d1a1961ba5a9d5de13d5d87ee740ddb9158dceb710d5f18b4eb230a8c92d"}
Sep 06 05:32:54.863836 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:54.863482    2578 scope.go:117] "RemoveContainer" containerID="de8c8dda54ab18ffc98523016fdd6c2efe839f914ea5d6bde5a6efd92598e74f"
Sep 06 05:32:54.884181 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:54.884148    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"]
Sep 06 05:32:54.887267 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:54.887245    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-5ec92654-kserve-7c647c4595-nqzp2"]
Sep 06 05:32:55.153914 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.153884    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40451dfc-2530-4ca8-a53f-bcb99a73b0f5" path="/var/lib/kubelet/pods/40451dfc-2530-4ca8-a53f-bcb99a73b0f5/volumes"
Sep 06 05:32:55.427587 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.427468    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r" podUID="55614266-7c57-4339-9a97-77abfa06b43e" containerName="main" containerID="cri-o://a23f4f19956634875a3f213487c194d09f5f6368ff9e905ac2eeeca2a192227d" gracePeriod=45
Sep 06 05:32:55.681567 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.681515    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:55.778886 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.778853    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55614266-7c57-4339-9a97-77abfa06b43e-tls-certs\") pod \"55614266-7c57-4339-9a97-77abfa06b43e\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") "
Sep 06 05:32:55.779086 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.778903    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-dshm\") pod \"55614266-7c57-4339-9a97-77abfa06b43e\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") "
Sep 06 05:32:55.779086 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.778955    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-model-cache\") pod \"55614266-7c57-4339-9a97-77abfa06b43e\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") "
Sep 06 05:32:55.779206 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.779078    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvxj5\" (UniqueName: \"kubernetes.io/projected/55614266-7c57-4339-9a97-77abfa06b43e-kube-api-access-zvxj5\") pod \"55614266-7c57-4339-9a97-77abfa06b43e\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") "
Sep 06 05:32:55.779206 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.779126    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-tmp-dir\") pod \"55614266-7c57-4339-9a97-77abfa06b43e\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") "
Sep 06 05:32:55.779206 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.779175    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-home\") pod \"55614266-7c57-4339-9a97-77abfa06b43e\" (UID: \"55614266-7c57-4339-9a97-77abfa06b43e\") "
Sep 06 05:32:55.779206 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.779183    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-model-cache" (OuterVolumeSpecName: "model-cache") pod "55614266-7c57-4339-9a97-77abfa06b43e" (UID: "55614266-7c57-4339-9a97-77abfa06b43e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:32:55.779425 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.779340    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "55614266-7c57-4339-9a97-77abfa06b43e" (UID: "55614266-7c57-4339-9a97-77abfa06b43e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:32:55.779473 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.779437    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-home" (OuterVolumeSpecName: "home") pod "55614266-7c57-4339-9a97-77abfa06b43e" (UID: "55614266-7c57-4339-9a97-77abfa06b43e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:32:55.779529 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.779514    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-model-cache\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:32:55.779583 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.779531    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-tmp-dir\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:32:55.779583 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.779540    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-home\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:32:55.781098 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.781075    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55614266-7c57-4339-9a97-77abfa06b43e-kube-api-access-zvxj5" (OuterVolumeSpecName: "kube-api-access-zvxj5") pod "55614266-7c57-4339-9a97-77abfa06b43e" (UID: "55614266-7c57-4339-9a97-77abfa06b43e"). InnerVolumeSpecName "kube-api-access-zvxj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:32:55.781546 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.781524    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-dshm" (OuterVolumeSpecName: "dshm") pod "55614266-7c57-4339-9a97-77abfa06b43e" (UID: "55614266-7c57-4339-9a97-77abfa06b43e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:32:55.781593 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.781524    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55614266-7c57-4339-9a97-77abfa06b43e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "55614266-7c57-4339-9a97-77abfa06b43e" (UID: "55614266-7c57-4339-9a97-77abfa06b43e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:32:55.869922 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.869888    2578 generic.go:356] "Generic (PLEG): container finished" podID="55614266-7c57-4339-9a97-77abfa06b43e" containerID="a23f4f19956634875a3f213487c194d09f5f6368ff9e905ac2eeeca2a192227d" exitCode=0
Sep 06 05:32:55.870312 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.869927    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"
Sep 06 05:32:55.870312 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.869980    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r" event={"ID":"55614266-7c57-4339-9a97-77abfa06b43e","Type":"ContainerDied","Data":"a23f4f19956634875a3f213487c194d09f5f6368ff9e905ac2eeeca2a192227d"}
Sep 06 05:32:55.870312 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.870033    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r" event={"ID":"55614266-7c57-4339-9a97-77abfa06b43e","Type":"ContainerDied","Data":"0cd5d122d81b725840a87ba378b68da440bc8e6f6068c04a6a491f33ab508bd6"}
Sep 06 05:32:55.870312 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.870057    2578 scope.go:117] "RemoveContainer" containerID="a23f4f19956634875a3f213487c194d09f5f6368ff9e905ac2eeeca2a192227d"
Sep 06 05:32:55.879316 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.879297    2578 scope.go:117] "RemoveContainer" containerID="a23f4f19956634875a3f213487c194d09f5f6368ff9e905ac2eeeca2a192227d"
Sep 06 05:32:55.879562 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:32:55.879546    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a23f4f19956634875a3f213487c194d09f5f6368ff9e905ac2eeeca2a192227d\": container with ID starting with a23f4f19956634875a3f213487c194d09f5f6368ff9e905ac2eeeca2a192227d not found: ID does not exist" containerID="a23f4f19956634875a3f213487c194d09f5f6368ff9e905ac2eeeca2a192227d"
Sep 06 05:32:55.879609 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.879573    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a23f4f19956634875a3f213487c194d09f5f6368ff9e905ac2eeeca2a192227d"} err="failed to get container status \"a23f4f19956634875a3f213487c194d09f5f6368ff9e905ac2eeeca2a192227d\": rpc error: code = NotFound desc = could not find container \"a23f4f19956634875a3f213487c194d09f5f6368ff9e905ac2eeeca2a192227d\": container with ID starting with a23f4f19956634875a3f213487c194d09f5f6368ff9e905ac2eeeca2a192227d not found: ID does not exist"
Sep 06 05:32:55.879952 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.879940    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55614266-7c57-4339-9a97-77abfa06b43e-tls-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:32:55.879989 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.879955    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55614266-7c57-4339-9a97-77abfa06b43e-dshm\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:32:55.879989 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.879964    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvxj5\" (UniqueName: \"kubernetes.io/projected/55614266-7c57-4339-9a97-77abfa06b43e-kube-api-access-zvxj5\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:32:55.889850 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.889817    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"]
Sep 06 05:32:55.893646 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:55.893623    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-5ec92654-kserve-748f44d58c-wxc5r"]
Sep 06 05:32:57.153956 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:32:57.153922    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55614266-7c57-4339-9a97-77abfa06b43e" path="/var/lib/kubelet/pods/55614266-7c57-4339-9a97-77abfa06b43e/volumes"
Sep 06 05:33:00.287670 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.287638    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"]
Sep 06 05:33:00.288068 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.288009    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40451dfc-2530-4ca8-a53f-bcb99a73b0f5" containerName="main"
Sep 06 05:33:00.288068 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.288035    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="40451dfc-2530-4ca8-a53f-bcb99a73b0f5" containerName="main"
Sep 06 05:33:00.288068 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.288049    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55614266-7c57-4339-9a97-77abfa06b43e" containerName="main"
Sep 06 05:33:00.288068 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.288055    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="55614266-7c57-4339-9a97-77abfa06b43e" containerName="main"
Sep 06 05:33:00.288288 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.288103    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="40451dfc-2530-4ca8-a53f-bcb99a73b0f5" containerName="main"
Sep 06 05:33:00.288288 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.288112    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="55614266-7c57-4339-9a97-77abfa06b43e" containerName="main"
Sep 06 05:33:00.292477 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.292455    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:00.303235 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.303204    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"]
Sep 06 05:33:00.318896 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.318863    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a807a5-6e38-4a25-803d-c0d3aef7fab1-maas-api-tls\") pod \"maas-api-6886844f88-qbs2m\" (UID: \"54a807a5-6e38-4a25-803d-c0d3aef7fab1\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:00.319078 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.318907    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a807a5-6e38-4a25-803d-c0d3aef7fab1-openshift-service-ca\") pod \"maas-api-6886844f88-qbs2m\" (UID: \"54a807a5-6e38-4a25-803d-c0d3aef7fab1\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:00.319078 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.318933    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc6hw\" (UniqueName: \"kubernetes.io/projected/54a807a5-6e38-4a25-803d-c0d3aef7fab1-kube-api-access-gc6hw\") pod \"maas-api-6886844f88-qbs2m\" (UID: \"54a807a5-6e38-4a25-803d-c0d3aef7fab1\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:00.319078 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.319051    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a807a5-6e38-4a25-803d-c0d3aef7fab1-metrics-certs\") pod \"maas-api-6886844f88-qbs2m\" (UID: \"54a807a5-6e38-4a25-803d-c0d3aef7fab1\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:00.420069 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.420008    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a807a5-6e38-4a25-803d-c0d3aef7fab1-metrics-certs\") pod \"maas-api-6886844f88-qbs2m\" (UID: \"54a807a5-6e38-4a25-803d-c0d3aef7fab1\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:00.420217 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.420155    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a807a5-6e38-4a25-803d-c0d3aef7fab1-maas-api-tls\") pod \"maas-api-6886844f88-qbs2m\" (UID: \"54a807a5-6e38-4a25-803d-c0d3aef7fab1\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:00.420217 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.420195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a807a5-6e38-4a25-803d-c0d3aef7fab1-openshift-service-ca\") pod \"maas-api-6886844f88-qbs2m\" (UID: \"54a807a5-6e38-4a25-803d-c0d3aef7fab1\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:00.420341 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.420219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc6hw\" (UniqueName: \"kubernetes.io/projected/54a807a5-6e38-4a25-803d-c0d3aef7fab1-kube-api-access-gc6hw\") pod \"maas-api-6886844f88-qbs2m\" (UID: \"54a807a5-6e38-4a25-803d-c0d3aef7fab1\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:00.420949 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.420915    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a807a5-6e38-4a25-803d-c0d3aef7fab1-openshift-service-ca\") pod \"maas-api-6886844f88-qbs2m\" (UID: \"54a807a5-6e38-4a25-803d-c0d3aef7fab1\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:00.422543 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.422512    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a807a5-6e38-4a25-803d-c0d3aef7fab1-metrics-certs\") pod \"maas-api-6886844f88-qbs2m\" (UID: \"54a807a5-6e38-4a25-803d-c0d3aef7fab1\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:00.422731 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.422713    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a807a5-6e38-4a25-803d-c0d3aef7fab1-maas-api-tls\") pod \"maas-api-6886844f88-qbs2m\" (UID: \"54a807a5-6e38-4a25-803d-c0d3aef7fab1\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:00.427186 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.427164    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc6hw\" (UniqueName: \"kubernetes.io/projected/54a807a5-6e38-4a25-803d-c0d3aef7fab1-kube-api-access-gc6hw\") pod \"maas-api-6886844f88-qbs2m\" (UID: \"54a807a5-6e38-4a25-803d-c0d3aef7fab1\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:00.605598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.605511    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:00.875212 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.875179    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"]
Sep 06 05:33:00.878332 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:33:00.878305    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a807a5_6e38_4a25_803d_c0d3aef7fab1.slice/crio-5a9953f4bbfaab60b1bf1ec57d75fff343ebd040b50fb3bd2c8f8c3343261359 WatchSource:0}: Error finding container 5a9953f4bbfaab60b1bf1ec57d75fff343ebd040b50fb3bd2c8f8c3343261359: Status 404 returned error can't find the container with id 5a9953f4bbfaab60b1bf1ec57d75fff343ebd040b50fb3bd2c8f8c3343261359
Sep 06 05:33:00.899842 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:00.899812    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m" event={"ID":"54a807a5-6e38-4a25-803d-c0d3aef7fab1","Type":"ContainerStarted","Data":"5a9953f4bbfaab60b1bf1ec57d75fff343ebd040b50fb3bd2c8f8c3343261359"}
Sep 06 05:33:01.910426 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:01.910371    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m" event={"ID":"54a807a5-6e38-4a25-803d-c0d3aef7fab1","Type":"ContainerStarted","Data":"1a74ff106263bd1f14f11d42dca4ee053226ef99f31bc9fdfb820942e5ae1f91"}
Sep 06 05:33:01.910936 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:01.910632    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:01.928610 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:01.928559    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m" podStartSLOduration=1.455833942 podStartE2EDuration="1.928545272s" totalImagesPullingTime="472.71133ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:33:01.927278117 +0000 UTC m=+2249.296733362" watchObservedRunningTime="2026-09-06 05:33:01.928545272 +0000 UTC m=+2249.298000518"
Sep 06 05:33:07.921388 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:07.921352    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6886844f88-qbs2m"
Sep 06 05:33:15.262265 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:15.262228    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"]
Sep 06 05:33:15.262713 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:15.262489    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv" podUID="28d975fb-da06-49c7-9a42-9a0970c403fb" containerName="maas-api" containerID="cri-o://f0d8a3e32b383111280e5a0f9dab508ec88640b343bf7e2a58702cbb350c29bd" gracePeriod=30
Sep 06 05:33:16.527711 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.527677    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:33:16.675157 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.675126    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28d975fb-da06-49c7-9a42-9a0970c403fb-metrics-certs\") pod \"28d975fb-da06-49c7-9a42-9a0970c403fb\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") "
Sep 06 05:33:16.675353 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.675169    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d975fb-da06-49c7-9a42-9a0970c403fb-openshift-service-ca\") pod \"28d975fb-da06-49c7-9a42-9a0970c403fb\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") "
Sep 06 05:33:16.675353 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.675193    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28d975fb-da06-49c7-9a42-9a0970c403fb-maas-api-tls\") pod \"28d975fb-da06-49c7-9a42-9a0970c403fb\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") "
Sep 06 05:33:16.675353 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.675241    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d95gp\" (UniqueName: \"kubernetes.io/projected/28d975fb-da06-49c7-9a42-9a0970c403fb-kube-api-access-d95gp\") pod \"28d975fb-da06-49c7-9a42-9a0970c403fb\" (UID: \"28d975fb-da06-49c7-9a42-9a0970c403fb\") "
Sep 06 05:33:16.675565 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.675538    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28d975fb-da06-49c7-9a42-9a0970c403fb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28d975fb-da06-49c7-9a42-9a0970c403fb" (UID: "28d975fb-da06-49c7-9a42-9a0970c403fb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:33:16.677377 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.677347    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28d975fb-da06-49c7-9a42-9a0970c403fb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "28d975fb-da06-49c7-9a42-9a0970c403fb" (UID: "28d975fb-da06-49c7-9a42-9a0970c403fb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:33:16.677495 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.677423    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28d975fb-da06-49c7-9a42-9a0970c403fb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28d975fb-da06-49c7-9a42-9a0970c403fb" (UID: "28d975fb-da06-49c7-9a42-9a0970c403fb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:33:16.677495 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.677458    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28d975fb-da06-49c7-9a42-9a0970c403fb-kube-api-access-d95gp" (OuterVolumeSpecName: "kube-api-access-d95gp") pod "28d975fb-da06-49c7-9a42-9a0970c403fb" (UID: "28d975fb-da06-49c7-9a42-9a0970c403fb"). InnerVolumeSpecName "kube-api-access-d95gp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:33:16.776691 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.776653    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28d975fb-da06-49c7-9a42-9a0970c403fb-metrics-certs\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:33:16.776691 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.776683    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d975fb-da06-49c7-9a42-9a0970c403fb-openshift-service-ca\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:33:16.776691 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.776697    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28d975fb-da06-49c7-9a42-9a0970c403fb-maas-api-tls\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:33:16.776934 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.776706    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d95gp\" (UniqueName: \"kubernetes.io/projected/28d975fb-da06-49c7-9a42-9a0970c403fb-kube-api-access-d95gp\") on node \"ip-10-0-134-29.ec2.internal\" DevicePath \"\""
Sep 06 05:33:16.988360 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.988272    2578 generic.go:356] "Generic (PLEG): container finished" podID="28d975fb-da06-49c7-9a42-9a0970c403fb" containerID="f0d8a3e32b383111280e5a0f9dab508ec88640b343bf7e2a58702cbb350c29bd" exitCode=0
Sep 06 05:33:16.988360 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.988317    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"
Sep 06 05:33:16.988597 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.988361    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv" event={"ID":"28d975fb-da06-49c7-9a42-9a0970c403fb","Type":"ContainerDied","Data":"f0d8a3e32b383111280e5a0f9dab508ec88640b343bf7e2a58702cbb350c29bd"}
Sep 06 05:33:16.988597 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.988423    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv" event={"ID":"28d975fb-da06-49c7-9a42-9a0970c403fb","Type":"ContainerDied","Data":"8c213478f7db8eb7c5d64e07813c8918d873aef51831dec0e02ee1fd57ddf466"}
Sep 06 05:33:16.988597 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.988453    2578 scope.go:117] "RemoveContainer" containerID="f0d8a3e32b383111280e5a0f9dab508ec88640b343bf7e2a58702cbb350c29bd"
Sep 06 05:33:16.998487 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.998468    2578 scope.go:117] "RemoveContainer" containerID="f0d8a3e32b383111280e5a0f9dab508ec88640b343bf7e2a58702cbb350c29bd"
Sep 06 05:33:16.998755 ip-10-0-134-29 kubenswrapper[2578]: E0906 05:33:16.998736    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0d8a3e32b383111280e5a0f9dab508ec88640b343bf7e2a58702cbb350c29bd\": container with ID starting with f0d8a3e32b383111280e5a0f9dab508ec88640b343bf7e2a58702cbb350c29bd not found: ID does not exist" containerID="f0d8a3e32b383111280e5a0f9dab508ec88640b343bf7e2a58702cbb350c29bd"
Sep 06 05:33:16.998819 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:16.998763    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0d8a3e32b383111280e5a0f9dab508ec88640b343bf7e2a58702cbb350c29bd"} err="failed to get container status \"f0d8a3e32b383111280e5a0f9dab508ec88640b343bf7e2a58702cbb350c29bd\": rpc error: code = NotFound desc = could not find container \"f0d8a3e32b383111280e5a0f9dab508ec88640b343bf7e2a58702cbb350c29bd\": container with ID starting with f0d8a3e32b383111280e5a0f9dab508ec88640b343bf7e2a58702cbb350c29bd not found: ID does not exist"
Sep 06 05:33:17.008704 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:17.008678    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"]
Sep 06 05:33:17.012140 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:17.012116    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b97c6dbfb-p9hvv"]
Sep 06 05:33:17.153182 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:17.153151    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28d975fb-da06-49c7-9a42-9a0970c403fb" path="/var/lib/kubelet/pods/28d975fb-da06-49c7-9a42-9a0970c403fb/volumes"
Sep 06 05:33:38.790043 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:38.790002    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gtn45_0b42b54e-dd87-427f-87e7-3620664c6f4e/manager/0.log"
Sep 06 05:33:38.907171 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:38.907136    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-g2nw4_cad8cce1-0dec-42c9-9b98-d706cc3ec37c/manager/0.log"
Sep 06 05:33:39.036458 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:39.036417    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8557bfdbbd-qzs94_9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3/manager/0.log"
Sep 06 05:33:39.643509 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:39.643483    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657c48fd97-wrlfn_1cbba029-1e85-41bc-8342-44a5c5c33b7f/copy-manifests/0.log"
Sep 06 05:33:39.657294 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:39.657267    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657c48fd97-wrlfn_1cbba029-1e85-41bc-8342-44a5c5c33b7f/manager/0.log"
Sep 06 05:33:40.107705 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:40.107613    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6886844f88-qbs2m_54a807a5-6e38-4a25-803d-c0d3aef7fab1/maas-api/0.log"
Sep 06 05:33:41.303339 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:41.303303    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-slkcz_4aa5fc66-cede-4224-bb27-d4e9f85000fe/manager/0.log"
Sep 06 05:33:41.413384 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:41.413355    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-ql9kj_b8c762b2-56e7-4071-a4d4-03d7651ee8ed/manager/0.log"
Sep 06 05:33:41.646179 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:41.646149    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-77kqr_368363d8-2c06-4076-98b0-dcb96679ddf9/manager/0.log"
Sep 06 05:33:41.881503 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:41.881470    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-g8q6h_10dbeb7b-8863-4952-be43-f23bb8533bab/manager/0.log"
Sep 06 05:33:42.219379 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:42.219350    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf_9a81cc22-70c0-49e9-a55a-9bfa627dcf5f/istio-proxy/0.log"
Sep 06 05:33:42.548976 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:42.548883    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c96787df8-ndm87_23793d3c-462d-4a32-a0aa-56cd314392a8/kube-auth-proxy/0.log"
Sep 06 05:33:42.666951 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:42.666921    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-s8dcv_58864663-ba1f-4c9a-bebd-0a016c980b43/istio-proxy/1.log"
Sep 06 05:33:42.907416 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:42.907381    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-b5x42_2c73f38f-cd1d-4b43-9f88-e7a06578fdfc/payload-processing/0.log"
Sep 06 05:33:43.931378 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:43.931338    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-p9dzz_262842d9-f0e8-4b50-ae64-00c6ac028505/main/0.log"
Sep 06 05:33:44.042701 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:44.042671    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-gjsg8_59927d2d-edf3-473b-a818-0f80ca82ac6b/main/0.log"
Sep 06 05:33:49.074875 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.074841    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5ns8q/must-gather-hdl8j"]
Sep 06 05:33:49.075273 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.075250    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28d975fb-da06-49c7-9a42-9a0970c403fb" containerName="maas-api"
Sep 06 05:33:49.075273 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.075262    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="28d975fb-da06-49c7-9a42-9a0970c403fb" containerName="maas-api"
Sep 06 05:33:49.075354 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.075317    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="28d975fb-da06-49c7-9a42-9a0970c403fb" containerName="maas-api"
Sep 06 05:33:49.078502 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.078485    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ns8q/must-gather-hdl8j"
Sep 06 05:33:49.080964 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.080943    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5ns8q\"/\"kube-root-ca.crt\""
Sep 06 05:33:49.081095 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.081075    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5ns8q\"/\"openshift-service-ca.crt\""
Sep 06 05:33:49.081146 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.081108    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5ns8q\"/\"default-dockercfg-p7szt\""
Sep 06 05:33:49.085896 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.085871    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ns8q/must-gather-hdl8j"]
Sep 06 05:33:49.088258 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.088212    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:33:49.147782 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.147743    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log" limit=1024
Sep 06 05:33:49.179110 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.179079    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b7324718-1a7f-4fa5-92b8-61fa1ba70892-must-gather-output\") pod \"must-gather-hdl8j\" (UID: \"b7324718-1a7f-4fa5-92b8-61fa1ba70892\") " pod="openshift-must-gather-5ns8q/must-gather-hdl8j"
Sep 06 05:33:49.179276 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.179172    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sjml\" (UniqueName: \"kubernetes.io/projected/b7324718-1a7f-4fa5-92b8-61fa1ba70892-kube-api-access-2sjml\") pod \"must-gather-hdl8j\" (UID: \"b7324718-1a7f-4fa5-92b8-61fa1ba70892\") " pod="openshift-must-gather-5ns8q/must-gather-hdl8j"
Sep 06 05:33:49.280373 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.280336    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sjml\" (UniqueName: \"kubernetes.io/projected/b7324718-1a7f-4fa5-92b8-61fa1ba70892-kube-api-access-2sjml\") pod \"must-gather-hdl8j\" (UID: \"b7324718-1a7f-4fa5-92b8-61fa1ba70892\") " pod="openshift-must-gather-5ns8q/must-gather-hdl8j"
Sep 06 05:33:49.280542 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.280406    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b7324718-1a7f-4fa5-92b8-61fa1ba70892-must-gather-output\") pod \"must-gather-hdl8j\" (UID: \"b7324718-1a7f-4fa5-92b8-61fa1ba70892\") " pod="openshift-must-gather-5ns8q/must-gather-hdl8j"
Sep 06 05:33:49.280713 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.280697    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b7324718-1a7f-4fa5-92b8-61fa1ba70892-must-gather-output\") pod \"must-gather-hdl8j\" (UID: \"b7324718-1a7f-4fa5-92b8-61fa1ba70892\") " pod="openshift-must-gather-5ns8q/must-gather-hdl8j"
Sep 06 05:33:49.288540 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.288509    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sjml\" (UniqueName: \"kubernetes.io/projected/b7324718-1a7f-4fa5-92b8-61fa1ba70892-kube-api-access-2sjml\") pod \"must-gather-hdl8j\" (UID: \"b7324718-1a7f-4fa5-92b8-61fa1ba70892\") " pod="openshift-must-gather-5ns8q/must-gather-hdl8j"
Sep 06 05:33:49.388491 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.388457    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ns8q/must-gather-hdl8j"
Sep 06 05:33:49.658168 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:49.658139    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ns8q/must-gather-hdl8j"]
Sep 06 05:33:49.659610 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:33:49.659582    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7324718_1a7f_4fa5_92b8_61fa1ba70892.slice/crio-291a5fb38b7d57c50df77db40103d344e200c041dc47b83acf2033667520d871 WatchSource:0}: Error finding container 291a5fb38b7d57c50df77db40103d344e200c041dc47b83acf2033667520d871: Status 404 returned error can't find the container with id 291a5fb38b7d57c50df77db40103d344e200c041dc47b83acf2033667520d871
Sep 06 05:33:50.160526 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:50.160491    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ns8q/must-gather-hdl8j" event={"ID":"b7324718-1a7f-4fa5-92b8-61fa1ba70892","Type":"ContainerStarted","Data":"291a5fb38b7d57c50df77db40103d344e200c041dc47b83acf2033667520d871"}
Sep 06 05:33:51.167868 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:51.167833    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ns8q/must-gather-hdl8j" event={"ID":"b7324718-1a7f-4fa5-92b8-61fa1ba70892","Type":"ContainerStarted","Data":"d499e2854c40b831dafa17b74832f9706fa60554ddf72c23bc57c65f5d6dce49"}
Sep 06 05:33:51.168274 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:51.167878    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ns8q/must-gather-hdl8j" event={"ID":"b7324718-1a7f-4fa5-92b8-61fa1ba70892","Type":"ContainerStarted","Data":"59e709b46a7176b2c0f643b435e7e9c3459ee38944cabfda19fc0aae2425ab20"}
Sep 06 05:33:51.182809 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:51.182765    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5ns8q/must-gather-hdl8j" podStartSLOduration=1.441680525 podStartE2EDuration="2.182749762s" totalImagesPullingTime="741.069237ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:33:51.181802118 +0000 UTC m=+2298.551257366" watchObservedRunningTime="2026-09-06 05:33:51.182749762 +0000 UTC m=+2298.552205008"
Sep 06 05:33:51.951568 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:51.951541    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n2jlj_d794e27a-146d-4d21-871c-3efaf67efeea/global-pull-secret-syncer/0.log"
Sep 06 05:33:52.028199 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:52.028161    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4bhll_f6516247-fb60-4bc3-9db3-6c90ac893019/konnectivity-agent/0.log"
Sep 06 05:33:52.198977 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:52.198948    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-29.ec2.internal_9ed8709ffad4ba828dceaf06e7817a74/haproxy/0.log"
Sep 06 05:33:56.926065 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:56.926010    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-slkcz_4aa5fc66-cede-4224-bb27-d4e9f85000fe/manager/0.log"
Sep 06 05:33:56.959634 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:56.959542    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-ql9kj_b8c762b2-56e7-4071-a4d4-03d7651ee8ed/manager/0.log"
Sep 06 05:33:57.020040 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:57.019944    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-77kqr_368363d8-2c06-4076-98b0-dcb96679ddf9/manager/0.log"
Sep 06 05:33:57.122659 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:57.122620    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-g8q6h_10dbeb7b-8863-4952-be43-f23bb8533bab/manager/0.log"
Sep 06 05:33:58.519300 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:58.519272    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4b76fbf2-0a15-4385-8a20-7728b4d37269/alertmanager/0.log"
Sep 06 05:33:58.538749 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:58.538720    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4b76fbf2-0a15-4385-8a20-7728b4d37269/config-reloader/0.log"
Sep 06 05:33:58.561700 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:58.561670    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4b76fbf2-0a15-4385-8a20-7728b4d37269/kube-rbac-proxy-web/0.log"
Sep 06 05:33:58.583178 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:58.583149    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4b76fbf2-0a15-4385-8a20-7728b4d37269/kube-rbac-proxy/0.log"
Sep 06 05:33:58.602676 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:58.602649    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4b76fbf2-0a15-4385-8a20-7728b4d37269/kube-rbac-proxy-metric/0.log"
Sep 06 05:33:58.621061 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:58.621038    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4b76fbf2-0a15-4385-8a20-7728b4d37269/prom-label-proxy/0.log"
Sep 06 05:33:58.640234 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:58.640186    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4b76fbf2-0a15-4385-8a20-7728b4d37269/init-config-reloader/0.log"
Sep 06 05:33:58.962723 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:58.962685    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vp8qm_422d64d1-f6c6-4b05-8149-304fe879eb59/node-exporter/0.log"
Sep 06 05:33:58.982372 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:58.982318    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vp8qm_422d64d1-f6c6-4b05-8149-304fe879eb59/kube-rbac-proxy/0.log"
Sep 06 05:33:59.000689 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:59.000654    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vp8qm_422d64d1-f6c6-4b05-8149-304fe879eb59/init-textfile/0.log"
Sep 06 05:33:59.351479 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:59.351392    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-l7kbk_e7a05bfc-b1b2-49d3-b48e-6e49a5a69581/prometheus-operator-admission-webhook/0.log"
Sep 06 05:33:59.458198 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:59.458172    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6696d6fcb6-9j4th_49d317ee-f7b8-4602-a9c5-486ad7a27a64/thanos-query/0.log"
Sep 06 05:33:59.477900 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:59.477865    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6696d6fcb6-9j4th_49d317ee-f7b8-4602-a9c5-486ad7a27a64/kube-rbac-proxy-web/0.log"
Sep 06 05:33:59.495963 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:59.495935    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6696d6fcb6-9j4th_49d317ee-f7b8-4602-a9c5-486ad7a27a64/kube-rbac-proxy/0.log"
Sep 06 05:33:59.516467 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:59.516435    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6696d6fcb6-9j4th_49d317ee-f7b8-4602-a9c5-486ad7a27a64/prom-label-proxy/0.log"
Sep 06 05:33:59.541303 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:59.541257    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6696d6fcb6-9j4th_49d317ee-f7b8-4602-a9c5-486ad7a27a64/kube-rbac-proxy-rules/0.log"
Sep 06 05:33:59.559347 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:33:59.559311    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6696d6fcb6-9j4th_49d317ee-f7b8-4602-a9c5-486ad7a27a64/kube-rbac-proxy-metrics/0.log"
Sep 06 05:34:02.074598 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.074568    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7fff8d58f8-95th4_bc4ea106-4f5d-4499-a964-564510515a9f/console/0.log"
Sep 06 05:34:02.107842 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.107809    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-8jjn2_0ce339fc-a3c7-40bf-8d50-384552a8da06/download-server/0.log"
Sep 06 05:34:02.160912 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.160866    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"]
Sep 06 05:34:02.165372 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.165342    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.172475 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.172443    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"]
Sep 06 05:34:02.318735 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.318703    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/427a5838-c309-4b3b-ad9b-f0fe8cb97668-proc\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.318935 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.318744    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/427a5838-c309-4b3b-ad9b-f0fe8cb97668-sys\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.318935 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.318769    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/427a5838-c309-4b3b-ad9b-f0fe8cb97668-podres\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.318935 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.318845    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5gfq\" (UniqueName: \"kubernetes.io/projected/427a5838-c309-4b3b-ad9b-f0fe8cb97668-kube-api-access-s5gfq\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.318935 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.318891    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/427a5838-c309-4b3b-ad9b-f0fe8cb97668-lib-modules\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.419818 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.419776    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/427a5838-c309-4b3b-ad9b-f0fe8cb97668-proc\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.420052 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.419838    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/427a5838-c309-4b3b-ad9b-f0fe8cb97668-sys\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.420052 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.419881    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/427a5838-c309-4b3b-ad9b-f0fe8cb97668-podres\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.420052 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.419917    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5gfq\" (UniqueName: \"kubernetes.io/projected/427a5838-c309-4b3b-ad9b-f0fe8cb97668-kube-api-access-s5gfq\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.420052 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.419946    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/427a5838-c309-4b3b-ad9b-f0fe8cb97668-lib-modules\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.420282 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.420182    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/427a5838-c309-4b3b-ad9b-f0fe8cb97668-lib-modules\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.420282 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.420241    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/427a5838-c309-4b3b-ad9b-f0fe8cb97668-proc\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.420386 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.420282    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/427a5838-c309-4b3b-ad9b-f0fe8cb97668-sys\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.420386 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.420354    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/427a5838-c309-4b3b-ad9b-f0fe8cb97668-podres\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.428860 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.428827    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5gfq\" (UniqueName: \"kubernetes.io/projected/427a5838-c309-4b3b-ad9b-f0fe8cb97668-kube-api-access-s5gfq\") pod \"perf-node-gather-daemonset-gdkwg\" (UID: \"427a5838-c309-4b3b-ad9b-f0fe8cb97668\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.480765 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.480731    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:02.777073 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:02.776980    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"]
Sep 06 05:34:02.781144 ip-10-0-134-29 kubenswrapper[2578]: W0906 05:34:02.781115    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod427a5838_c309_4b3b_ad9b_f0fe8cb97668.slice/crio-c707aea3f994a4682613cbe8c332fbd663ab6c2c41faeec6bf77bfbb4f7f58e4 WatchSource:0}: Error finding container c707aea3f994a4682613cbe8c332fbd663ab6c2c41faeec6bf77bfbb4f7f58e4: Status 404 returned error can't find the container with id c707aea3f994a4682613cbe8c332fbd663ab6c2c41faeec6bf77bfbb4f7f58e4
Sep 06 05:34:03.264429 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:03.264378    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg" event={"ID":"427a5838-c309-4b3b-ad9b-f0fe8cb97668","Type":"ContainerStarted","Data":"4f3571cfbe24ca947df69ef86523bb13ff28470cb82ab50f1558667693cff5ed"}
Sep 06 05:34:03.264911 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:03.264442    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg" event={"ID":"427a5838-c309-4b3b-ad9b-f0fe8cb97668","Type":"ContainerStarted","Data":"c707aea3f994a4682613cbe8c332fbd663ab6c2c41faeec6bf77bfbb4f7f58e4"}
Sep 06 05:34:03.264911 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:03.264470    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:03.280130 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:03.280066    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg" podStartSLOduration=1.280043536 podStartE2EDuration="1.280043536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 05:34:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:34:03.27831125 +0000 UTC m=+2310.647766497" watchObservedRunningTime="2026-09-06 05:34:03.280043536 +0000 UTC m=+2310.649498781"
Sep 06 05:34:03.944333 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:03.944297    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jzb7s_379e032e-b73c-47bf-ad03-faa4ffc31425/dns/0.log"
Sep 06 05:34:03.967579 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:03.967547    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jzb7s_379e032e-b73c-47bf-ad03-faa4ffc31425/kube-rbac-proxy/0.log"
Sep 06 05:34:04.068601 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:04.068565    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-79mhk_f5adf721-8d7d-4791-8410-575e3dfa6541/dns-node-resolver/0.log"
Sep 06 05:34:04.624668 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:04.624639    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-97hkw_ed8f6484-9497-4655-853a-1b79956c4145/node-ca/0.log"
Sep 06 05:34:05.511575 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:05.511542    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf_9a81cc22-70c0-49e9-a55a-9bfa627dcf5f/istio-proxy/0.log"
Sep 06 05:34:05.839246 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:05.839160    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c96787df8-ndm87_23793d3c-462d-4a32-a0aa-56cd314392a8/kube-auth-proxy/0.log"
Sep 06 05:34:05.865421 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:05.865388    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-s8dcv_58864663-ba1f-4c9a-bebd-0a016c980b43/istio-proxy/0.log"
Sep 06 05:34:05.884772 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:05.884724    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-s8dcv_58864663-ba1f-4c9a-bebd-0a016c980b43/istio-proxy/1.log"
Sep 06 05:34:05.964251 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:05.964220    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-b5x42_2c73f38f-cd1d-4b43-9f88-e7a06578fdfc/payload-processing/0.log"
Sep 06 05:34:06.826750 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:06.826716    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mdzll_abb6c0e7-871b-4b97-9a23-906c01495e8a/serve-healthcheck-canary/0.log"
Sep 06 05:34:07.505185 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:07.505157    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vkvc2_4d78dc73-4a38-43c7-865a-c74aeca7478f/kube-rbac-proxy/0.log"
Sep 06 05:34:07.523980 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:07.523952    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vkvc2_4d78dc73-4a38-43c7-865a-c74aeca7478f/exporter/0.log"
Sep 06 05:34:07.567029 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:07.566987    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vkvc2_4d78dc73-4a38-43c7-865a-c74aeca7478f/extractor/0.log"
Sep 06 05:34:09.281123 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:09.281091    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-gdkwg"
Sep 06 05:34:09.596008 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:09.595936    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gtn45_0b42b54e-dd87-427f-87e7-3620664c6f4e/manager/0.log"
Sep 06 05:34:09.928141 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:09.928096    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-g2nw4_cad8cce1-0dec-42c9-9b98-d706cc3ec37c/manager/0.log"
Sep 06 05:34:09.992242 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:09.992208    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8557bfdbbd-qzs94_9ca03c2c-e39e-4086-b06c-f6a5bbd04ec3/manager/0.log"
Sep 06 05:34:10.237651 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:10.237573    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657c48fd97-wrlfn_1cbba029-1e85-41bc-8342-44a5c5c33b7f/manager/0.log"
Sep 06 05:34:10.256162 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:10.256135    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657c48fd97-wrlfn_1cbba029-1e85-41bc-8342-44a5c5c33b7f/copy-manifests/0.log"
Sep 06 05:34:11.642925 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:11.642887    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-6bj2f_0fef7531-decc-4a4d-ad24-c22669ee08e7/openshift-lws-operator/0.log"
Sep 06 05:34:12.863068 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:12.863035    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf_9a81cc22-70c0-49e9-a55a-9bfa627dcf5f/istio-proxy/0.log"
Sep 06 05:34:13.156190 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:13.156164    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c96787df8-ndm87_23793d3c-462d-4a32-a0aa-56cd314392a8/kube-auth-proxy/0.log"
Sep 06 05:34:13.181073 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:13.181040    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-s8dcv_58864663-ba1f-4c9a-bebd-0a016c980b43/istio-proxy/0.log"
Sep 06 05:34:13.199051 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:13.198999    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-s8dcv_58864663-ba1f-4c9a-bebd-0a016c980b43/istio-proxy/1.log"
Sep 06 05:34:13.261265 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:13.261230    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-b5x42_2c73f38f-cd1d-4b43-9f88-e7a06578fdfc/payload-processing/0.log"
Sep 06 05:34:14.872627 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:14.872566    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-d69pf_9a81cc22-70c0-49e9-a55a-9bfa627dcf5f/istio-proxy/0.log"
Sep 06 05:34:15.186081 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:15.185983    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c96787df8-ndm87_23793d3c-462d-4a32-a0aa-56cd314392a8/kube-auth-proxy/0.log"
Sep 06 05:34:15.211339 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:15.211301    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-s8dcv_58864663-ba1f-4c9a-bebd-0a016c980b43/istio-proxy/0.log"
Sep 06 05:34:15.221067 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:15.221039    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-s8dcv_58864663-ba1f-4c9a-bebd-0a016c980b43/istio-proxy/1.log"
Sep 06 05:34:15.287682 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:15.287651    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-b5x42_2c73f38f-cd1d-4b43-9f88-e7a06578fdfc/payload-processing/0.log"
Sep 06 05:34:18.235059 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:18.235008    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jgkf2_7143ad84-9eb1-4336-9ba4-48b6f73a8ee3/kube-multus-additional-cni-plugins/0.log"
Sep 06 05:34:18.253571 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:18.253541    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jgkf2_7143ad84-9eb1-4336-9ba4-48b6f73a8ee3/egress-router-binary-copy/0.log"
Sep 06 05:34:18.270527 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:18.270504    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jgkf2_7143ad84-9eb1-4336-9ba4-48b6f73a8ee3/cni-plugins/0.log"
Sep 06 05:34:18.290258 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:18.290236    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jgkf2_7143ad84-9eb1-4336-9ba4-48b6f73a8ee3/bond-cni-plugin/0.log"
Sep 06 05:34:18.307776 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:18.307752    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jgkf2_7143ad84-9eb1-4336-9ba4-48b6f73a8ee3/routeoverride-cni/0.log"
Sep 06 05:34:18.325209 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:18.325186    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jgkf2_7143ad84-9eb1-4336-9ba4-48b6f73a8ee3/whereabouts-cni-bincopy/0.log"
Sep 06 05:34:18.344849 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:18.344824    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jgkf2_7143ad84-9eb1-4336-9ba4-48b6f73a8ee3/whereabouts-cni/0.log"
Sep 06 05:34:18.533562 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:18.533490    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-j6jvp_837ae00e-1c4e-4e9c-8170-d7c018203862/kube-multus/0.log"
Sep 06 05:34:18.642241 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:18.642213    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dxn9s_d85cbcaa-00ae-4440-b7f0-7209f409a5cd/network-metrics-daemon/0.log"
Sep 06 05:34:18.655121 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:18.655098    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dxn9s_d85cbcaa-00ae-4440-b7f0-7209f409a5cd/kube-rbac-proxy/0.log"
Sep 06 05:34:19.882501 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:19.882468    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zfz7w_953a51c8-56ba-4c1c-9333-fb4cf193f4d6/ovn-controller/0.log"
Sep 06 05:34:19.896733 ip-10-0-134-29 kubenswrapper[2578]: I0906 05:34:19.896702    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zfz7w_953a51c8-56ba-4c1c-9333-fb4cf193f4d6/ovn-acl-logging/0.log"