Jul 16 15:33:30.588098 ip-10-0-141-84 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 16 15:33:30.588109 ip-10-0-141-84 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 16 15:33:30.588116 ip-10-0-141-84 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 16 15:33:30.588348 ip-10-0-141-84 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 16 15:33:40.643936 ip-10-0-141-84 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 16 15:33:40.643950 ip-10-0-141-84 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 4ea6e096bcb64f6f858eeed3ff81a293 --
Jul 16 15:35:33.597188 ip-10-0-141-84 systemd[1]: Starting Kubernetes Kubelet...
Jul 16 15:35:34.047566 ip-10-0-141-84 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 15:35:34.047566 ip-10-0-141-84 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 16 15:35:34.047566 ip-10-0-141-84 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 15:35:34.047566 ip-10-0-141-84 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 16 15:35:34.047566 ip-10-0-141-84 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 15:35:34.048639 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.048504    2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 16 15:35:34.053133 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053118    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 15:35:34.053133 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053132    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053136    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053140    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053145    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053149    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053152    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053156    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053159    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053162    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053164    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053168    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053171    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053175    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053178    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053180    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053183    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053185    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053188    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053190    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 15:35:34.053198 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053193    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053202    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053206    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053208    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053211    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053214    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053216    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053219    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053222    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053224    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053227    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053229    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053232    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053235    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053237    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053240    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053242    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053245    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053247    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053250    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 15:35:34.053637 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053252    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053254    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053257    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053259    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053262    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053264    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053266    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053269    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053271    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053274    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053276    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053279    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053281    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053284    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053288    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053290    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053293    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053295    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053298    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053300    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 15:35:34.054120 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053303    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053305    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053307    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053310    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053313    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053315    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053317    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053320    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053322    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053325    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053329    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053331    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053334    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053336    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053339    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053343    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053347    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053349    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053352    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 15:35:34.054588 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053354    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053357    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053360    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053362    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053364    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053367    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053369    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053372    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053375    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053378    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053380    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053383    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053385    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053388    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053390    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053393    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053395    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053397    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053400    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.053403    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 15:35:34.055068 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054080    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054087    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054090    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054093    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054097    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054099    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054102    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054105    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054107    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054111    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054115    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054118    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054121    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054124    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054126    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054129    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054131    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054134    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054137    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 15:35:34.055538 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054140    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054142    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054146    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054149    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054152    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054154    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054157    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054159    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054162    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054165    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054167    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054170    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054172    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054175    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054177    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054180    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054182    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054185    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054188    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054191    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 15:35:34.056015 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054193    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054196    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054198    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054201    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054203    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054206    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054208    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054211    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054214    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054218    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054221    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054225    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054228    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054231    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054235    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054239    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054243    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054246    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054249    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 15:35:34.056476 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054252    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054255    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054257    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054260    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054262    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054265    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054267    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054270    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054272    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054275    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054277    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054280    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054283    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054286    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054288    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054290    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054293    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054295    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054298    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054300    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 15:35:34.056943 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054302    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054305    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054307    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054310    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054312    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054316    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054319    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054321    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054324    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054327    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054329    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054332    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054335    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054337    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054340    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054343    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054345    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054347    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054350    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 15:35:34.057426 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054352    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054355    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054700    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054706    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054709    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054712    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054714    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054717    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054720    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054723    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054726    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054728    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054731    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054734    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054736    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054739    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054741    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054744    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054747    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 15:35:34.057894 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054749    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054752    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054755    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054758    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054761    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054763    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054766    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054769    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054772    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054774    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054777    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054780    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054782    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054785    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054787    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054790    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054792    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054795    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054797    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054800    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 15:35:34.058371 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054804    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054808    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054811    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054814    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054817    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054820    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054823    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054825    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054828    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054830    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054833    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054835    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054838    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054841    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054843    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054846    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054849    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054851    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054853    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054856    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 15:35:34.058853 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054858    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054860    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054863    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054865    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054868    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054870    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054872    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054876    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054878    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054881    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054883    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054886    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054888    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054891    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054893    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054896    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054898    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054901    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054903    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054906    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 15:35:34.059395 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054909    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054911    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054914    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054916    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054919    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054921    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054923    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054926    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054941    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054949    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054951    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054954    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054957    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054960    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054962    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054972    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054975    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054978    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054981    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054983    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 15:35:34.059864 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054988    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.054991    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055074    2570 flags.go:64] FLAG: --address="0.0.0.0"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055081    2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055087    2570 flags.go:64] FLAG: --anonymous-auth="true"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055092    2570 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055098    2570 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055102    2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055107    2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055111    2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055115    2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055118    2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055122    2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055125    2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055128    2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055131    2570 flags.go:64] FLAG: --cgroup-root=""
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055134    2570 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055137    2570 flags.go:64] FLAG: --client-ca-file=""
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055140    2570 flags.go:64] FLAG: --cloud-provider="external"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055143    2570 flags.go:64] FLAG: --cluster-dns="[]"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055147    2570 flags.go:64] FLAG: --cluster-domain=""
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055150    2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055154    2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055157    2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 16 15:35:34.060372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055160    2570 flags.go:64] FLAG: --container-log-max-files="5"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055164    2570 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055167    2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055170    2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055173    2570 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055176    2570 flags.go:64] FLAG: --contention-profiling="false"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055179    2570 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055182    2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055185    2570 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055188    2570 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055191    2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055194    2570 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055197    2570 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055200    2570 flags.go:64] FLAG: --enable-load-reader="false"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055204    2570 flags.go:64] FLAG: --enable-server="true"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055207    2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055211    2570 flags.go:64] FLAG: --event-burst="100"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055215    2570 flags.go:64] FLAG: --event-qps="50"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055218    2570 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055222    2570 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055225    2570 flags.go:64] FLAG: --eviction-hard=""
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055228    2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055231    2570 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055234    2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055237    2570 flags.go:64] FLAG: --eviction-soft=""
Jul 16 15:35:34.060982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055240    2570 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055243    2570 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055246    2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055249    2570 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055252    2570 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055254    2570 flags.go:64] FLAG: --fail-swap-on="true"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055258    2570 flags.go:64] FLAG: --feature-gates=""
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055261    2570 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055264    2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055267    2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055270    2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055273    2570 flags.go:64] FLAG: --healthz-port="10248"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055276    2570 flags.go:64] FLAG: --help="false"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055279    2570 flags.go:64] FLAG: --hostname-override="ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055282    2570 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055285    2570 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055288    2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055291    2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055295    2570 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055298    2570 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055300    2570 flags.go:64] FLAG: --image-service-endpoint=""
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055304    2570 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055307    2570 flags.go:64] FLAG: --kube-api-burst="100"
Jul 16 15:35:34.061649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055312    2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055315    2570 flags.go:64] FLAG: --kube-api-qps="50"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055318    2570 flags.go:64] FLAG: --kube-reserved=""
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055321    2570 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055328    2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055331    2570 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055334    2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055337    2570 flags.go:64] FLAG: --lock-file=""
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055339    2570 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055342    2570 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055346    2570 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055351    2570 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055354    2570 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055357    2570 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055359    2570 flags.go:64] FLAG: --logging-format="text"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055363    2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055366    2570 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055369    2570 flags.go:64] FLAG: --manifest-url=""
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055372    2570 flags.go:64] FLAG: --manifest-url-header=""
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055376    2570 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055379    2570 flags.go:64] FLAG: --max-open-files="1000000"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055383    2570 flags.go:64] FLAG: --max-pods="110"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055386    2570 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055389    2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055392    2570 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 16 15:35:34.062615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055395    2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055398    2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055401    2570 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055404    2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055410    2570 flags.go:64] FLAG: --node-status-max-images="50"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055414    2570 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055417    2570 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055422    2570 flags.go:64] FLAG: --pod-cidr=""
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055425    2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055430    2570 flags.go:64] FLAG: --pod-manifest-path=""
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055433    2570 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055436    2570 flags.go:64] FLAG: --pods-per-core="0"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055439    2570 flags.go:64] FLAG: --port="10250"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055442    2570 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055445    2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bf1248ac4a21ea45"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055448    2570 flags.go:64] FLAG: --qos-reserved=""
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055451    2570 flags.go:64] FLAG: --read-only-port="10255"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055454    2570 flags.go:64] FLAG: --register-node="true"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055457    2570 flags.go:64] FLAG: --register-with-taints=""
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055460    2570 flags.go:64] FLAG: --registry-burst="10"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055463    2570 flags.go:64] FLAG: --registry-qps="5"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055466    2570 flags.go:64] FLAG: --reserved-cpus=""
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055468    2570 flags.go:64] FLAG: --reserved-memory=""
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055472    2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055476    2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 16 15:35:34.063498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055479    2570 flags.go:64] FLAG: --rotate-certificates="false"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055482    2570 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055485    2570 flags.go:64] FLAG: --runonce="false"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055488    2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055491    2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055493    2570 flags.go:64] FLAG: --seccomp-default="false"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055496    2570 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055499    2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055502    2570 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055505    2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055508    2570 flags.go:64] FLAG: --storage-driver-password="root"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055511    2570 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055514    2570 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055518    2570 flags.go:64] FLAG: --storage-driver-user="root"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055523    2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055526    2570 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055529    2570 flags.go:64] FLAG: --system-cgroups=""
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055532    2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055538    2570 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055541    2570 flags.go:64] FLAG: --tls-cert-file=""
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055544    2570 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055547    2570 flags.go:64] FLAG: --tls-min-version=""
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055550    2570 flags.go:64] FLAG: --tls-private-key-file=""
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055553    2570 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055556    2570 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 16 15:35:34.064140 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055559    2570 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055562    2570 flags.go:64] FLAG: --v="2"
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055566    2570 flags.go:64] FLAG: --version="false"
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055570    2570 flags.go:64] FLAG: --vmodule=""
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055574    2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.055577    2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055687    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055691    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055695    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055698    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055701    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055704    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055708    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055712    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055715    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055718    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055721    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055723    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055726    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055729    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 15:35:34.064783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055732    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055735    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055738    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055740    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055743    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055746    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055748    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055750    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055753    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055756    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055758    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055761    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055763    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055766    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055768    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055771    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055774    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055776    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055779    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055781    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 15:35:34.065285 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055787    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055790    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055792    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055795    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055797    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055800    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055802    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055804    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055807    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055809    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055812    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055814    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055817    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055822    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055825    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055828    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055831    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055833    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055836    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055838    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 15:35:34.065760 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055841    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055843    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055846    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055848    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055850    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055853    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055855    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055858    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055860    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055863    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055866    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055868    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055871    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055874    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055877    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055879    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055881    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055884    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055887    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055890    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 15:35:34.066259 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055892    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055895    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055897    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055900    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055903    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055907    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055910    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055913    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055916    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055918    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055921    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055923    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055925    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055941    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055944    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055946    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055949    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055952    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055954    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055957    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 15:35:34.066731 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055959    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055962    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055964    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055967    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.055970    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.056505    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.064557    2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.064571    2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.065362    2570 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.065370    2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065426    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065431    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065434    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065437    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065440    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 15:35:34.067230 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065443    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065446    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065451    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065455    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065458    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065461    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065464    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065467    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065469    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065472    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065476    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065478    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065481    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065483    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065486    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065488    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065491    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065494    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065496    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 15:35:34.067606 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065499    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065502    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065505    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065507    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065510    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065512    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065515    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065518    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065521    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065523    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065526    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065528    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065531    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065534    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065536    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065539    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065542    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065544    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065547    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065550    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 15:35:34.068073 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065552    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065555    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065559    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065561    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065564    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065566    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065569    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065571    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065575    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065578    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065581    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065584    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065587    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065591    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065593    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065596    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065599    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065602    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065604    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 15:35:34.068638 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065607    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065609    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065612    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065614    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065617    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065619    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065622    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065624    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065627    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065630    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065632    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065635    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065637    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065640    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065642    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065645    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065649    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065651    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065653    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065656    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 15:35:34.069103 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065659    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065661    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065664    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065666    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065670    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065674    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065677    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065680    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065683    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065685    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065687    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065690    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065692    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065695    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065697    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 15:35:34.069572 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065700    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.065705    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065812    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065817    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065820    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065823    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065826    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065829    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065850    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065854    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065857    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065860    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065863    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065865    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065869    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 15:35:34.069942 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065872    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065875    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065879    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065883    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065886    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065889    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065893    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065896    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065899    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065902    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065905    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065907    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065910    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065912    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065915    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065918    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065920    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065923    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065925    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 15:35:34.070335 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065941    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065944    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065947    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065950    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065953    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065955    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065958    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065961    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065964    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065966    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065969    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065972    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065974    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065979    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065981    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065984    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065986    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065990    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065993    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065996    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 15:35:34.070783 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.065999    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066002    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066005    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066008    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066010    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066012    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066015    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066018    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066021    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066025    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066028    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066030    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066033    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066035    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066038    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066040    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066043    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066045    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066048    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066050    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 15:35:34.071282 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066053    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066055    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066058    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066060    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066063    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066065    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066068    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066071    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066073    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066076    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066078    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066081    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066083    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066086    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066088    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066091    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066093    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066095    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066098    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066101    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 15:35:34.071747 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066103    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 15:35:34.072236 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066106    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 15:35:34.072236 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066108    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 15:35:34.072236 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066111    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 15:35:34.072236 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066114    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 15:35:34.072236 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066116    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 15:35:34.072236 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:34.066119    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 15:35:34.072236 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.066123    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 15:35:34.072236 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.066219    2570 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 16 15:35:34.072236 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.070579    2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 16 15:35:34.072236 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.072154    2570 server.go:1019] "Starting client certificate rotation"
Jul 16 15:35:34.072490 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.072258    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 15:35:34.072490 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.072300    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 15:35:34.074342 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.074328    2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 15:35:34.075516 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.075502    2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 15:35:34.085837 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.085814    2570 log.go:25] "Validated CRI v1 runtime API"
Jul 16 15:35:34.087904 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.087887    2570 log.go:25] "Validated CRI v1 image API"
Jul 16 15:35:34.088688 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.088669    2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 16 15:35:34.093516 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.093492    2570 fs.go:135] Filesystem UUIDs: map[3b0d28b3-be53-4bf2-9906-f375027da642:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 bb4c9536-7369-470f-b114-06e8708539b0:/dev/nvme0n1p4]
Jul 16 15:35:34.093579 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.093515    2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 16 15:35:34.098285 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.098186    2570 manager.go:216] Machine: {Timestamp:2026-07-16 15:35:34.096962664 +0000 UTC m=+0.285826630 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3111872 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d0abbeda1c22af2576fee39d1dc9a SystemUUID:ec2d0abb-eda1-c22a-f257-6fee39d1dc9a BootID:4ea6e096-bcb6-4f6f-858e-eed3ff81a293 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b6:66:0c:52:bb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b6:66:0c:52:bb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:ac:79:a7:09:4b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 16 15:35:34.098285 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.098281    2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 16 15:35:34.098392 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.098348    2570 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 16 15:35:34.101179 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.101156    2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 16 15:35:34.101302 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.101182    2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-84.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 16 15:35:34.101796 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.101785    2570 topology_manager.go:138] "Creating topology manager with none policy"
Jul 16 15:35:34.101826 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.101800    2570 container_manager_linux.go:309] "Creating device plugin manager"
Jul 16 15:35:34.101826 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.101808    2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 15:35:34.101876 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.101854    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 15:35:34.102569 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.102559    2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 15:35:34.103939 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.103919    2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 16 15:35:34.107876 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.107864    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 15:35:34.108495 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.108464    2570 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 16 15:35:34.108615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.108510    2570 kubelet.go:486] "Attempting to sync node with API server"
Jul 16 15:35:34.108615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.108543    2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 16 15:35:34.108615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.108564    2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 16 15:35:34.108615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.108582    2570 kubelet.go:398] "Adding apiserver pod source"
Jul 16 15:35:34.108615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.108604    2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 16 15:35:34.109261 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.109035    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 15:35:34.109261 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.109061    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 15:35:34.111765 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.111747    2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 16 15:35:34.113587 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.113574    2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 16 15:35:34.113638 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.113604    2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 16 15:35:34.113762 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.113753    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 16 15:35:34.113793 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.113768    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 16 15:35:34.113793 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.113774    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 16 15:35:34.113793 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.113779    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 16 15:35:34.113793 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.113787    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 16 15:35:34.113793 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.113792    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 16 15:35:34.114165 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.114156    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 16 15:35:34.114165 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.114165    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 16 15:35:34.114228 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.114172    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 16 15:35:34.114228 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.114178    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 16 15:35:34.114228 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.114187    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 16 15:35:34.114228 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.114194    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 16 15:35:34.114806 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.114798    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 16 15:35:34.114806 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.114806    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 16 15:35:34.118070 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.118053    2570 server.go:1268] "Started kubelet"
Jul 16 15:35:34.118169 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.118122    2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 16 15:35:34.118310 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.118257    2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 16 15:35:34.118440 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.118330    2570 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 16 15:35:34.119301 ip-10-0-141-84 systemd[1]: Started Kubernetes Kubelet.
Jul 16 15:35:34.119733 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.119715    2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 16 15:35:34.120222 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.120203    2570 server.go:310] "Adding debug handlers to kubelet server"
Jul 16 15:35:34.120461 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.120442    2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-84.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 16 15:35:34.120975 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.120951    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 16 15:35:34.120975 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.120960    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-84.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 16 15:35:34.121590 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.121568    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hvjjz"
Jul 16 15:35:34.123191 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.123160    2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 16 15:35:34.124783 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.124766    2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 16 15:35:34.124783 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.124780    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 16 15:35:34.124906 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.124877    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:34.125418 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.125349    2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 16 15:35:34.125418 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.125349    2570 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 16 15:35:34.125418 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.125376    2570 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 16 15:35:34.125621 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.125490    2570 factory.go:55] Registering systemd factory
Jul 16 15:35:34.125621 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.125510    2570 reconstruct.go:97] "Volume reconstruction finished"
Jul 16 15:35:34.125621 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.125519    2570 reconciler.go:29] "Reconciler: start to sync state"
Jul 16 15:35:34.125621 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.125514    2570 factory.go:223] Registration of the systemd container factory successfully
Jul 16 15:35:34.125822 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.125720    2570 factory.go:153] Registering CRI-O factory
Jul 16 15:35:34.125822 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.125736    2570 factory.go:223] Registration of the crio container factory successfully
Jul 16 15:35:34.125822 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.125786    2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 16 15:35:34.125822 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.125809    2570 factory.go:103] Registering Raw factory
Jul 16 15:35:34.125822 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.125825    2570 manager.go:1196] Started watching for new ooms in manager
Jul 16 15:35:34.126349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.126334    2570 manager.go:349] Starting recovery of all containers
Jul 16 15:35:34.126613 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.125733    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-84.ec2.internal.18c2cf1eeca01d50  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-84.ec2.internal,UID:ip-10-0-141-84.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-84.ec2.internal,},FirstTimestamp:2026-07-16 15:35:34.11802248 +0000 UTC m=+0.306886428,LastTimestamp:2026-07-16 15:35:34.11802248 +0000 UTC m=+0.306886428,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-84.ec2.internal,}"
Jul 16 15:35:34.131623 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.130033    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hvjjz"
Jul 16 15:35:34.132907 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.132655    2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-84.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 16 15:35:34.132907 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.132745    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 16 15:35:34.137530 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.137513    2570 manager.go:354] Recovery completed
Jul 16 15:35:34.143089 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.143077    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 15:35:34.145017 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.144999    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 15:35:34.145105 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.145034    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 15:35:34.145105 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.145048    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasSufficientPID"
Jul 16 15:35:34.145617 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.145605    2570 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 16 15:35:34.145617 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.145615    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 15:35:34.145724 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.145617    2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 16 15:35:34.145724 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.145689    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 15:35:34.146298 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.146282    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 15:35:34.146351 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.146307    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 15:35:34.146351 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.146316    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasSufficientPID"
Jul 16 15:35:34.147735 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.147722    2570 policy_none.go:49] "None policy: Start"
Jul 16 15:35:34.147789 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.147750    2570 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 16 15:35:34.147789 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.147765    2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 16 15:35:34.149809 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.149795    2570 policy_none.go:47] "Start"
Jul 16 15:35:34.188658 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.188541    2570 manager.go:337] "Starting Device Plugin manager"
Jul 16 15:35:34.188773 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.188694    2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 16 15:35:34.188773 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.188708    2570 server.go:87] "Starting device plugin registration server"
Jul 16 15:35:34.188957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.188938    2570 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 16 15:35:34.189015 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.188953    2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 16 15:35:34.189101 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.189081    2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 16 15:35:34.189180 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.189167    2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 16 15:35:34.189180 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.189180    2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 16 15:35:34.189569 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.189552    2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 16 15:35:34.189640 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.189604    2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:34.259428 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.259404    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 16 15:35:34.261655 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.260607    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 16 15:35:34.261655 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.260639    2570 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 16 15:35:34.261655 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.260659    2570 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 16 15:35:34.261655 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.260695    2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 16 15:35:34.263122 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.263102    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 15:35:34.289123 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.289096    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 15:35:34.289918 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.289900    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 15:35:34.290006 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.289948    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 15:35:34.290006 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.289964    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasSufficientPID"
Jul 16 15:35:34.290006 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.289986    2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.299864 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.299806    2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.299864 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.299826    2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-84.ec2.internal\": node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:34.315529 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.315507    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:34.361088 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.361069    2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-84.ec2.internal"]
Jul 16 15:35:34.361161 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.361131    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 15:35:34.362527 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.362513    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 15:35:34.362593 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.362541    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 15:35:34.362593 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.362551    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasSufficientPID"
Jul 16 15:35:34.364927 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.364915    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 15:35:34.365049 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.365027    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.365106 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.365064    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 15:35:34.365603 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.365584    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 15:35:34.365603 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.365598    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 15:35:34.365748 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.365619    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 15:35:34.365748 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.365633    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasSufficientPID"
Jul 16 15:35:34.365748 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.365618    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 15:35:34.365748 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.365698    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasSufficientPID"
Jul 16 15:35:34.368428 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.368410    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.368508 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.368444    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 15:35:34.369139 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.369122    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 15:35:34.369236 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.369149    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 15:35:34.369236 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.369158    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeHasSufficientPID"
Jul 16 15:35:34.389556 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.389534    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-84.ec2.internal\" not found" node="ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.393832 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.393816    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-84.ec2.internal\" not found" node="ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.415942 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.415916    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:34.426558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.426541    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/445e0b74d7f76eb893ad5f0460043313-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal\" (UID: \"445e0b74d7f76eb893ad5f0460043313\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.426609 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.426564    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/445e0b74d7f76eb893ad5f0460043313-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal\" (UID: \"445e0b74d7f76eb893ad5f0460043313\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.426609 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.426587    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/391d2eed45bf7533effc099da20934d8-config\") pod \"kube-apiserver-proxy-ip-10-0-141-84.ec2.internal\" (UID: \"391d2eed45bf7533effc099da20934d8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.516864 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.516843    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:34.527502 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.527485    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/391d2eed45bf7533effc099da20934d8-config\") pod \"kube-apiserver-proxy-ip-10-0-141-84.ec2.internal\" (UID: \"391d2eed45bf7533effc099da20934d8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.527552 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.527506    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/445e0b74d7f76eb893ad5f0460043313-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal\" (UID: \"445e0b74d7f76eb893ad5f0460043313\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.527552 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.527524    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/445e0b74d7f76eb893ad5f0460043313-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal\" (UID: \"445e0b74d7f76eb893ad5f0460043313\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.527618 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.527569    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/445e0b74d7f76eb893ad5f0460043313-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal\" (UID: \"445e0b74d7f76eb893ad5f0460043313\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.527618 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.527575    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/391d2eed45bf7533effc099da20934d8-config\") pod \"kube-apiserver-proxy-ip-10-0-141-84.ec2.internal\" (UID: \"391d2eed45bf7533effc099da20934d8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.527618 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.527590    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/445e0b74d7f76eb893ad5f0460043313-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal\" (UID: \"445e0b74d7f76eb893ad5f0460043313\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.617856 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.617791    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:34.692790 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.692764    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.696295 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:34.696265    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:34.718437 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.718409    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:34.820532 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.820493    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:34.923451 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:34.923362    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:35.024272 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:35.024248    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:35.039655 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:35.039635    2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 15:35:35.071462 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:35.071437    2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 16 15:35:35.071835 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:35.071552    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 15:35:35.071835 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:35.071555    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 15:35:35.125395 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:35.125372    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:35.134923 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:35.134893    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 15:30:34 +0000 UTC" deadline="2027-12-28 13:05:38.083802053 +0000 UTC"
Jul 16 15:35:35.134923 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:35.134915    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12717h30m2.948889743s"
Jul 16 15:35:35.148747 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:35.148726    2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 15:35:35.225922 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:35.225860    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:35.326261 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:35.326222    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:35.426632 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:35.426598    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-84.ec2.internal\" not found"
Jul 16 15:35:35.520300 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:35.520229    2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 15:35:35.525285 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:35.525270    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:35.536110 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:35.536089    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 15:35:35.537026 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:35.537014    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-84.ec2.internal"
Jul 16 15:35:35.545410 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:35.545397    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 15:35:35.708032 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:35.708002    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod445e0b74d7f76eb893ad5f0460043313.slice/crio-eb248a941c961ab37397ff8e222a7cfc8b1dc2ecd3735bde4fe6a85899006764 WatchSource:0}: Error finding container eb248a941c961ab37397ff8e222a7cfc8b1dc2ecd3735bde4fe6a85899006764: Status 404 returned error can't find the container with id eb248a941c961ab37397ff8e222a7cfc8b1dc2ecd3735bde4fe6a85899006764
Jul 16 15:35:35.713451 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:35.713434    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 15:35:36.110874 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.110842    2570 apiserver.go:52] "Watching apiserver"
Jul 16 15:35:36.118872 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.118850    2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 16 15:35:36.119210 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.119190    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-vk7rr","openshift-ovn-kubernetes/ovnkube-node-sltmp","kube-system/kube-apiserver-proxy-ip-10-0-141-84.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69","openshift-image-registry/node-ca-ftrpc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal","openshift-multus/multus-additional-cni-plugins-wc65r","openshift-network-diagnostics/network-check-target-8jdhr","kube-system/global-pull-secret-syncer-4t8xm","kube-system/konnectivity-agent-b4cff","openshift-cluster-node-tuning-operator/tuned-49wgv","openshift-dns/node-resolver-kqh5g","openshift-multus/multus-xv6dq","openshift-multus/network-metrics-daemon-98ggn"]
Jul 16 15:35:36.122815 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.122798    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.124891 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.124869    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.126061 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.126043    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 16 15:35:36.127218 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.127202    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:36.127312 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.127264    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:35:36.129389 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.129371    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.131518 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.131499    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ftrpc"
Jul 16 15:35:36.132579 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.132555    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 16 15:35:36.132680 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.132643    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 16 15:35:36.133707 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.133690    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:36.133803 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.133768    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:35:36.134487 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.134470    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 16 15:35:36.134591 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.134502    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 16 15:35:36.134591 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.134522    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 16 15:35:36.134814 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.134798    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8v9z6\""
Jul 16 15:35:36.134873 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.134832    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 16 15:35:36.134920 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.134910    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kmfgh\""
Jul 16 15:35:36.135315 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.135300    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 16 15:35:36.135887 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.135872    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:36.135977 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.135954    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:35:36.136247 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.136230    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 16 15:35:36.136330 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.136318    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n2f7p\""
Jul 16 15:35:36.136459 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.136439    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 16 15:35:36.136523 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.136470    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 16 15:35:36.136575 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.136559    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vkbkj\""
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.138145    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7be4b188-7ce2-4eba-8936-425c93d1fdad-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.138491    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vk7rr"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.138460    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-multus-socket-dir-parent\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.138538    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-hostroot\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.138569    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7be4b188-7ce2-4eba-8936-425c93d1fdad-cnibin\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.138600    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-system-cni-dir\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.138630    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-cnibin\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.138680    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-var-lib-cni-bin\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.138726    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-var-lib-kubelet\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.138917    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-multus-daemon-config\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.138989    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7be4b188-7ce2-4eba-8936-425c93d1fdad-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139056    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7be4b188-7ce2-4eba-8936-425c93d1fdad-system-cni-dir\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139145    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7be4b188-7ce2-4eba-8936-425c93d1fdad-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139221    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-multus-conf-dir\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139263    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs\") pod \"network-metrics-daemon-98ggn\" (UID: \"52710c6b-b4c4-48db-9637-2f94d8c959bb\") " pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139295    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7be4b188-7ce2-4eba-8936-425c93d1fdad-os-release\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.140877 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139375    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-os-release\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.141656 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139408    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdz7l\" (UniqueName: \"kubernetes.io/projected/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-kube-api-access-kdz7l\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.141656 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139458    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-run-multus-certs\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.141656 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139498    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88p8x\" (UniqueName: \"kubernetes.io/projected/7be4b188-7ce2-4eba-8936-425c93d1fdad-kube-api-access-88p8x\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.141656 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139534    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-multus-cni-dir\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.141656 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139571    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-cni-binary-copy\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.141656 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139638    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-run-k8s-cni-cncf-io\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.141656 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139669    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-run-netns\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.141656 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139703    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-var-lib-cni-multus\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.141656 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139728    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-etc-kubernetes\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.141656 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139763    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7be4b188-7ce2-4eba-8936-425c93d1fdad-cni-binary-copy\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.141656 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.139800    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grfhb\" (UniqueName: \"kubernetes.io/projected/52710c6b-b4c4-48db-9637-2f94d8c959bb-kube-api-access-grfhb\") pod \"network-metrics-daemon-98ggn\" (UID: \"52710c6b-b4c4-48db-9637-2f94d8c959bb\") " pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:36.142185 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.142124    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.142511 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.142492    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 16 15:35:36.142839 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.142826    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 16 15:35:36.143060 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.143048    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 16 15:35:36.143105 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.143083    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 16 15:35:36.143400 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.143387    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 16 15:35:36.144387 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.144372    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b4cff"
Jul 16 15:35:36.144834 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.144818    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jbnh9\""
Jul 16 15:35:36.146644 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.146627    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.148859 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.148842    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kqh5g"
Jul 16 15:35:36.150299 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.150282    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 16 15:35:36.151441 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.151419    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 16 15:35:36.151581 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.151456    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 16 15:35:36.151581 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.151433    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gl22h\""
Jul 16 15:35:36.151581 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.151516    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 16 15:35:36.151581 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.151559    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 16 15:35:36.152721 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.152687    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 16 15:35:36.152791 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.152778    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-scbd2\""
Jul 16 15:35:36.152989 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.152972    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 16 15:35:36.153117 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.152991    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 16 15:35:36.153212 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.153008    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 16 15:35:36.153212 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.153008    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 16 15:35:36.153212 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.153178    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ld2nd\""
Jul 16 15:35:36.153212 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.153082    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 16 15:35:36.153376 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.153087    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m2gxb\""
Jul 16 15:35:36.153528 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.153515    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 16 15:35:36.156654 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.156635    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 15:35:36.177362 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.177344    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vw6tf"
Jul 16 15:35:36.194359 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.194339    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vw6tf"
Jul 16 15:35:36.226918 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.226904    2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 16 15:35:36.240104 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240082    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wjn8\" (UniqueName: \"kubernetes.io/projected/d00c5a27-627f-40bc-9ca7-48b091ff51c9-kube-api-access-4wjn8\") pod \"iptables-alerter-vk7rr\" (UID: \"d00c5a27-627f-40bc-9ca7-48b091ff51c9\") " pod="openshift-network-operator/iptables-alerter-vk7rr"
Jul 16 15:35:36.240104 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240104    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730-agent-certs\") pod \"konnectivity-agent-b4cff\" (UID: \"51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730\") " pod="kube-system/konnectivity-agent-b4cff"
Jul 16 15:35:36.240242 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240119    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfbzm\" (UniqueName: \"kubernetes.io/projected/71ee235c-ac9d-418f-a4b5-cab77e7f5404-kube-api-access-mfbzm\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.240242 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240138    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7be4b188-7ce2-4eba-8936-425c93d1fdad-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.240242 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240171    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-hostroot\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.240242 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240200    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-slash\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.240436 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240242    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-run-systemd\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.240436 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240280    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb36e980-e40a-42e8-b1db-f032b02d683a-ovnkube-script-lib\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.240436 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240299    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-hostroot\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.240436 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240307    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-lib-modules\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.240436 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240336    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-system-cni-dir\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.240436 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240366    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-cnibin\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.240436 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240389    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-system-cni-dir\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.240436 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240392    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-var-lib-cni-bin\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.240436 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240407    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-cnibin\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.240847 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240433    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-var-lib-kubelet\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.240847 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240476    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-multus-daemon-config\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.240847 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240506    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-var-lib-kubelet\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.240847 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240506    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb36e980-e40a-42e8-b1db-f032b02d683a-ovn-node-metrics-cert\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.240847 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240536    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-tmp\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.240847 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240562    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcqwh\" (UniqueName: \"kubernetes.io/projected/fb36e980-e40a-42e8-b1db-f032b02d683a-kube-api-access-bcqwh\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.240847 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240587    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-var-lib-cni-bin\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.240847 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240619    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ww8x\" (UniqueName: \"kubernetes.io/projected/6a32f024-c920-4db1-8c81-37477f8dc910-kube-api-access-4ww8x\") pod \"node-resolver-kqh5g\" (UID: \"6a32f024-c920-4db1-8c81-37477f8dc910\") " pod="openshift-dns/node-resolver-kqh5g"
Jul 16 15:35:36.240847 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240643    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-sysconfig\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.240847 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240665    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7be4b188-7ce2-4eba-8936-425c93d1fdad-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.240847 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240667    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-sysctl-d\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.240847 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240719    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7be4b188-7ce2-4eba-8936-425c93d1fdad-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.240847 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240797    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-dbus\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:36.240847 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240838    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-os-release\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240904    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-os-release\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240908    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7be4b188-7ce2-4eba-8936-425c93d1fdad-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240906    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-cni-bin\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.240994    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241009    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-multus-daemon-config\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241020    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb36e980-e40a-42e8-b1db-f032b02d683a-ovnkube-config\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241077    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88p8x\" (UniqueName: \"kubernetes.io/projected/7be4b188-7ce2-4eba-8936-425c93d1fdad-kube-api-access-88p8x\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241101    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-multus-cni-dir\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241137    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-etc-openvswitch\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241160    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a32f024-c920-4db1-8c81-37477f8dc910-tmp-dir\") pod \"node-resolver-kqh5g\" (UID: \"6a32f024-c920-4db1-8c81-37477f8dc910\") " pod="openshift-dns/node-resolver-kqh5g"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241197    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-sysctl-conf\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241297    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-etc-kubernetes\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241302    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-multus-cni-dir\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241331    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7be4b188-7ce2-4eba-8936-425c93d1fdad-cni-binary-copy\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241334    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-etc-kubernetes\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241354    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-run-netns\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.241448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241375    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-sys\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241397    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-var-lib-kubelet\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241426    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grfhb\" (UniqueName: \"kubernetes.io/projected/52710c6b-b4c4-48db-9637-2f94d8c959bb-kube-api-access-grfhb\") pod \"network-metrics-daemon-98ggn\" (UID: \"52710c6b-b4c4-48db-9637-2f94d8c959bb\") " pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241452    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-device-dir\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241476    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-node-log\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241500    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/764123aa-64ce-40d5-9d5f-8be36c2026f6-serviceca\") pod \"node-ca-ftrpc\" (UID: \"764123aa-64ce-40d5-9d5f-8be36c2026f6\") " pod="openshift-image-registry/node-ca-ftrpc"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241534    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2fnt\" (UniqueName: \"kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt\") pod \"network-check-target-8jdhr\" (UID: \"958875c8-4f2f-474f-8201-b9090b15b099\") " pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241564    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d00c5a27-627f-40bc-9ca7-48b091ff51c9-host-slash\") pod \"iptables-alerter-vk7rr\" (UID: \"d00c5a27-627f-40bc-9ca7-48b091ff51c9\") " pod="openshift-network-operator/iptables-alerter-vk7rr"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241589    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-systemd\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241615    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-multus-socket-dir-parent\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241641    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-socket-dir\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241664    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-systemd-units\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241706    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb36e980-e40a-42e8-b1db-f032b02d683a-env-overrides\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241723    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7be4b188-7ce2-4eba-8936-425c93d1fdad-cni-binary-copy\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241728    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-run\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241762    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7be4b188-7ce2-4eba-8936-425c93d1fdad-cnibin\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.242187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241761    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-multus-socket-dir-parent\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241778    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-etc-selinux\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241793    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-sys-fs\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241814    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-host\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241819    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7be4b188-7ce2-4eba-8936-425c93d1fdad-cnibin\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241852    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7be4b188-7ce2-4eba-8936-425c93d1fdad-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241895    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-kubelet-config\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.241976    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-kubelet\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242000    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-var-lib-openvswitch\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242045    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-log-socket\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242080    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-kubernetes\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242116    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-tuned\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242141    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7be4b188-7ce2-4eba-8936-425c93d1fdad-system-cni-dir\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242175    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-multus-conf-dir\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242196    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs\") pod \"network-metrics-daemon-98ggn\" (UID: \"52710c6b-b4c4-48db-9637-2f94d8c959bb\") " pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242238    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7be4b188-7ce2-4eba-8936-425c93d1fdad-system-cni-dir\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242238    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:36.242799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242261    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-multus-conf-dir\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242297    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-run-openvswitch\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242309    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7be4b188-7ce2-4eba-8936-425c93d1fdad-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.242364    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242394    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a32f024-c920-4db1-8c81-37477f8dc910-hosts-file\") pod \"node-resolver-kqh5g\" (UID: \"6a32f024-c920-4db1-8c81-37477f8dc910\") " pod="openshift-dns/node-resolver-kqh5g"
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.242423    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs podName:52710c6b-b4c4-48db-9637-2f94d8c959bb nodeName:}" failed. No retries permitted until 2026-07-16 15:35:36.74239962 +0000 UTC m=+2.931263581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs") pod "network-metrics-daemon-98ggn" (UID: "52710c6b-b4c4-48db-9637-2f94d8c959bb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242447    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7b86\" (UniqueName: \"kubernetes.io/projected/764123aa-64ce-40d5-9d5f-8be36c2026f6-kube-api-access-n7b86\") pod \"node-ca-ftrpc\" (UID: \"764123aa-64ce-40d5-9d5f-8be36c2026f6\") " pod="openshift-image-registry/node-ca-ftrpc"
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242464    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730-konnectivity-ca\") pod \"konnectivity-agent-b4cff\" (UID: \"51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730\") " pod="kube-system/konnectivity-agent-b4cff"
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242489    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7be4b188-7ce2-4eba-8936-425c93d1fdad-os-release\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242534    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdz7l\" (UniqueName: \"kubernetes.io/projected/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-kube-api-access-kdz7l\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242567    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242582    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7be4b188-7ce2-4eba-8936-425c93d1fdad-os-release\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242594    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d00c5a27-627f-40bc-9ca7-48b091ff51c9-iptables-alerter-script\") pod \"iptables-alerter-vk7rr\" (UID: \"d00c5a27-627f-40bc-9ca7-48b091ff51c9\") " pod="openshift-network-operator/iptables-alerter-vk7rr"
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242612    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-modprobe-d\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242660    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-run-multus-certs\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242676    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-cni-binary-copy\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.243558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242690    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-run-k8s-cni-cncf-io\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.244320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242710    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-run-netns\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.244320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242736    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-run-multus-certs\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.244320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.242816    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-run-netns\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.244320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.243168    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-run-k8s-cni-cncf-io\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.244320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.243237    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-var-lib-cni-multus\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.244320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.243269    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-run-ovn-kubernetes\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.244320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.243307    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-cni-netd\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.244320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.243342    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnzvm\" (UniqueName: \"kubernetes.io/projected/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-kube-api-access-qnzvm\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.244320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.243375    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-registration-dir\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.244320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.243408    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-run-ovn\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.244320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.243425    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-host-var-lib-cni-multus\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.244320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.243457    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/764123aa-64ce-40d5-9d5f-8be36c2026f6-host\") pod \"node-ca-ftrpc\" (UID: \"764123aa-64ce-40d5-9d5f-8be36c2026f6\") " pod="openshift-image-registry/node-ca-ftrpc"
Jul 16 15:35:36.244320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.243555    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-cni-binary-copy\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.245962 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.245926    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 15:35:36.249029 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.249000    2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 16 15:35:36.252164 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.252146    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdz7l\" (UniqueName: \"kubernetes.io/projected/c1a2a201-9a63-40aa-bc2c-84b68a6a6a24-kube-api-access-kdz7l\") pod \"multus-xv6dq\" (UID: \"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24\") " pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.252227 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.252205    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grfhb\" (UniqueName: \"kubernetes.io/projected/52710c6b-b4c4-48db-9637-2f94d8c959bb-kube-api-access-grfhb\") pod \"network-metrics-daemon-98ggn\" (UID: \"52710c6b-b4c4-48db-9637-2f94d8c959bb\") " pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:36.253003 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.252983    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88p8x\" (UniqueName: \"kubernetes.io/projected/7be4b188-7ce2-4eba-8936-425c93d1fdad-kube-api-access-88p8x\") pod \"multus-additional-cni-plugins-wc65r\" (UID: \"7be4b188-7ce2-4eba-8936-425c93d1fdad\") " pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.264613 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.264581    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal" event={"ID":"445e0b74d7f76eb893ad5f0460043313","Type":"ContainerStarted","Data":"eb248a941c961ab37397ff8e222a7cfc8b1dc2ecd3735bde4fe6a85899006764"}
Jul 16 15:35:36.274333 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:36.274315    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod391d2eed45bf7533effc099da20934d8.slice/crio-605bb9980db064a5dc15b022929f79eddd908c1fe22ef0dfb0734a8b60266833 WatchSource:0}: Error finding container 605bb9980db064a5dc15b022929f79eddd908c1fe22ef0dfb0734a8b60266833: Status 404 returned error can't find the container with id 605bb9980db064a5dc15b022929f79eddd908c1fe22ef0dfb0734a8b60266833
Jul 16 15:35:36.344038 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344012    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-cni-bin\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.344168 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344047    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.344168 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344070    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb36e980-e40a-42e8-b1db-f032b02d683a-ovnkube-config\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.344168 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344091    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-etc-openvswitch\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.344168 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344112    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a32f024-c920-4db1-8c81-37477f8dc910-tmp-dir\") pod \"node-resolver-kqh5g\" (UID: \"6a32f024-c920-4db1-8c81-37477f8dc910\") " pod="openshift-dns/node-resolver-kqh5g"
Jul 16 15:35:36.344168 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344117    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-cni-bin\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.344168 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344127    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.344168 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344134    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-sysctl-conf\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344176    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-etc-openvswitch\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344183    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-run-netns\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344201    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-sys\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344217    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-var-lib-kubelet\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344233    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-device-dir\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344256    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-node-log\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344250    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-sysctl-conf\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344290    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-run-netns\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344304    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-node-log\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344331    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-sys\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344367    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-var-lib-kubelet\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344395    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-device-dir\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344434    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/764123aa-64ce-40d5-9d5f-8be36c2026f6-serviceca\") pod \"node-ca-ftrpc\" (UID: \"764123aa-64ce-40d5-9d5f-8be36c2026f6\") " pod="openshift-image-registry/node-ca-ftrpc"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344471    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2fnt\" (UniqueName: \"kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt\") pod \"network-check-target-8jdhr\" (UID: \"958875c8-4f2f-474f-8201-b9090b15b099\") " pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:36.344505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344495    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d00c5a27-627f-40bc-9ca7-48b091ff51c9-host-slash\") pod \"iptables-alerter-vk7rr\" (UID: \"d00c5a27-627f-40bc-9ca7-48b091ff51c9\") " pod="openshift-network-operator/iptables-alerter-vk7rr"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344544    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d00c5a27-627f-40bc-9ca7-48b091ff51c9-host-slash\") pod \"iptables-alerter-vk7rr\" (UID: \"d00c5a27-627f-40bc-9ca7-48b091ff51c9\") " pod="openshift-network-operator/iptables-alerter-vk7rr"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344668    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-systemd\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344708    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-socket-dir\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344734    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-systemd-units\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344758    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb36e980-e40a-42e8-b1db-f032b02d683a-env-overrides\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344779    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-run\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344737    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-systemd\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344803    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-etc-selinux\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344817    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-socket-dir\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344821    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-systemd-units\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344825    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-sys-fs\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344858    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-host\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344870    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/764123aa-64ce-40d5-9d5f-8be36c2026f6-serviceca\") pod \"node-ca-ftrpc\" (UID: \"764123aa-64ce-40d5-9d5f-8be36c2026f6\") " pod="openshift-image-registry/node-ca-ftrpc"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344881    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-kubelet-config\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344875    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-sys-fs\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344900    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-run\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344911    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-host\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.345124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344906    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-kubelet\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344924    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-etc-selinux\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344953    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-kubelet\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344961    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-kubelet-config\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344969    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-var-lib-openvswitch\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.344994    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-log-socket\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345009    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-kubernetes\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345023    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-tuned\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345062    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-log-socket\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345074    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-kubernetes\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345093    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-var-lib-openvswitch\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345097    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345166    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a32f024-c920-4db1-8c81-37477f8dc910-tmp-dir\") pod \"node-resolver-kqh5g\" (UID: \"6a32f024-c920-4db1-8c81-37477f8dc910\") " pod="openshift-dns/node-resolver-kqh5g"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.345182    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345185    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-run-openvswitch\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345223    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a32f024-c920-4db1-8c81-37477f8dc910-hosts-file\") pod \"node-resolver-kqh5g\" (UID: \"6a32f024-c920-4db1-8c81-37477f8dc910\") " pod="openshift-dns/node-resolver-kqh5g"
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.345244    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret podName:83adbeaf-0de4-4fec-be65-5a9a8b9ce326 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:36.845228253 +0000 UTC m=+3.034092192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret") pod "global-pull-secret-syncer-4t8xm" (UID: "83adbeaf-0de4-4fec-be65-5a9a8b9ce326") : object "kube-system"/"original-pull-secret" not registered
Jul 16 15:35:36.346002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345264    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-run-openvswitch\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345274    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7b86\" (UniqueName: \"kubernetes.io/projected/764123aa-64ce-40d5-9d5f-8be36c2026f6-kube-api-access-n7b86\") pod \"node-ca-ftrpc\" (UID: \"764123aa-64ce-40d5-9d5f-8be36c2026f6\") " pod="openshift-image-registry/node-ca-ftrpc"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345298    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb36e980-e40a-42e8-b1db-f032b02d683a-env-overrides\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345268    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a32f024-c920-4db1-8c81-37477f8dc910-hosts-file\") pod \"node-resolver-kqh5g\" (UID: \"6a32f024-c920-4db1-8c81-37477f8dc910\") " pod="openshift-dns/node-resolver-kqh5g"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345303    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730-konnectivity-ca\") pod \"konnectivity-agent-b4cff\" (UID: \"51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730\") " pod="kube-system/konnectivity-agent-b4cff"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345330    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb36e980-e40a-42e8-b1db-f032b02d683a-ovnkube-config\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345414    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345442    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d00c5a27-627f-40bc-9ca7-48b091ff51c9-iptables-alerter-script\") pod \"iptables-alerter-vk7rr\" (UID: \"d00c5a27-627f-40bc-9ca7-48b091ff51c9\") " pod="openshift-network-operator/iptables-alerter-vk7rr"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345466    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-modprobe-d\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345472    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345495    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-run-ovn-kubernetes\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345518    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-cni-netd\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345542    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnzvm\" (UniqueName: \"kubernetes.io/projected/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-kube-api-access-qnzvm\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345564    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-registration-dir\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345589    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-run-ovn\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345611    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/764123aa-64ce-40d5-9d5f-8be36c2026f6-host\") pod \"node-ca-ftrpc\" (UID: \"764123aa-64ce-40d5-9d5f-8be36c2026f6\") " pod="openshift-image-registry/node-ca-ftrpc"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345636    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wjn8\" (UniqueName: \"kubernetes.io/projected/d00c5a27-627f-40bc-9ca7-48b091ff51c9-kube-api-access-4wjn8\") pod \"iptables-alerter-vk7rr\" (UID: \"d00c5a27-627f-40bc-9ca7-48b091ff51c9\") " pod="openshift-network-operator/iptables-alerter-vk7rr"
Jul 16 15:35:36.346957 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345659    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730-agent-certs\") pod \"konnectivity-agent-b4cff\" (UID: \"51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730\") " pod="kube-system/konnectivity-agent-b4cff"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345682    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfbzm\" (UniqueName: \"kubernetes.io/projected/71ee235c-ac9d-418f-a4b5-cab77e7f5404-kube-api-access-mfbzm\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345727    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-slash\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345751    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-run-systemd\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345775    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb36e980-e40a-42e8-b1db-f032b02d683a-ovnkube-script-lib\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345797    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-lib-modules\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345825    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb36e980-e40a-42e8-b1db-f032b02d683a-ovn-node-metrics-cert\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345848    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-tmp\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345872    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcqwh\" (UniqueName: \"kubernetes.io/projected/fb36e980-e40a-42e8-b1db-f032b02d683a-kube-api-access-bcqwh\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345887    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730-konnectivity-ca\") pod \"konnectivity-agent-b4cff\" (UID: \"51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730\") " pod="kube-system/konnectivity-agent-b4cff"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345896    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ww8x\" (UniqueName: \"kubernetes.io/projected/6a32f024-c920-4db1-8c81-37477f8dc910-kube-api-access-4ww8x\") pod \"node-resolver-kqh5g\" (UID: \"6a32f024-c920-4db1-8c81-37477f8dc910\") " pod="openshift-dns/node-resolver-kqh5g"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345921    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-sysconfig\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345971    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d00c5a27-627f-40bc-9ca7-48b091ff51c9-iptables-alerter-script\") pod \"iptables-alerter-vk7rr\" (UID: \"d00c5a27-627f-40bc-9ca7-48b091ff51c9\") " pod="openshift-network-operator/iptables-alerter-vk7rr"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.345967    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-sysctl-d\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.346016    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-dbus\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.346047    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-modprobe-d\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.346098    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-run-ovn-kubernetes\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.347600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.346139    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-cni-netd\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.348349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.346143    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-sysctl-d\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.348349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.346201    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-host-slash\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.348349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.346413    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/764123aa-64ce-40d5-9d5f-8be36c2026f6-host\") pod \"node-ca-ftrpc\" (UID: \"764123aa-64ce-40d5-9d5f-8be36c2026f6\") " pod="openshift-image-registry/node-ca-ftrpc"
Jul 16 15:35:36.348349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.346738    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-dbus\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:36.348349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.346789    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-sysconfig\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.348349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.346894    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-lib-modules\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.348349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.346969    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71ee235c-ac9d-418f-a4b5-cab77e7f5404-registration-dir\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.348349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.346990    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-run-ovn\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.348349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.347042    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb36e980-e40a-42e8-b1db-f032b02d683a-run-systemd\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.348349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.347299    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-etc-tuned\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.348349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.347332    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb36e980-e40a-42e8-b1db-f032b02d683a-ovnkube-script-lib\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.348349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.348319    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb36e980-e40a-42e8-b1db-f032b02d683a-ovn-node-metrics-cert\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.348899 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.348654    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730-agent-certs\") pod \"konnectivity-agent-b4cff\" (UID: \"51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730\") " pod="kube-system/konnectivity-agent-b4cff"
Jul 16 15:35:36.348899 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.348681    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-tmp\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.352333 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.352317    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 15:35:36.352333 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.352336    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 15:35:36.352484 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.352345    2570 projected.go:196] Error preparing data for projected volume kube-api-access-n2fnt for pod openshift-network-diagnostics/network-check-target-8jdhr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 15:35:36.352484 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.352423    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt podName:958875c8-4f2f-474f-8201-b9090b15b099 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:36.852404487 +0000 UTC m=+3.041268445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n2fnt" (UniqueName: "kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt") pod "network-check-target-8jdhr" (UID: "958875c8-4f2f-474f-8201-b9090b15b099") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 15:35:36.354410 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.354390    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7b86\" (UniqueName: \"kubernetes.io/projected/764123aa-64ce-40d5-9d5f-8be36c2026f6-kube-api-access-n7b86\") pod \"node-ca-ftrpc\" (UID: \"764123aa-64ce-40d5-9d5f-8be36c2026f6\") " pod="openshift-image-registry/node-ca-ftrpc"
Jul 16 15:35:36.356341 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.356321    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfbzm\" (UniqueName: \"kubernetes.io/projected/71ee235c-ac9d-418f-a4b5-cab77e7f5404-kube-api-access-mfbzm\") pod \"aws-ebs-csi-driver-node-4sh69\" (UID: \"71ee235c-ac9d-418f-a4b5-cab77e7f5404\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.356517 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.356493    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnzvm\" (UniqueName: \"kubernetes.io/projected/eea902f9-fb25-4b75-94ce-bce7bd7a41d9-kube-api-access-qnzvm\") pod \"tuned-49wgv\" (UID: \"eea902f9-fb25-4b75-94ce-bce7bd7a41d9\") " pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.356995 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.356975    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ww8x\" (UniqueName: \"kubernetes.io/projected/6a32f024-c920-4db1-8c81-37477f8dc910-kube-api-access-4ww8x\") pod \"node-resolver-kqh5g\" (UID: \"6a32f024-c920-4db1-8c81-37477f8dc910\") " pod="openshift-dns/node-resolver-kqh5g"
Jul 16 15:35:36.357297 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.357236    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wjn8\" (UniqueName: \"kubernetes.io/projected/d00c5a27-627f-40bc-9ca7-48b091ff51c9-kube-api-access-4wjn8\") pod \"iptables-alerter-vk7rr\" (UID: \"d00c5a27-627f-40bc-9ca7-48b091ff51c9\") " pod="openshift-network-operator/iptables-alerter-vk7rr"
Jul 16 15:35:36.358402 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.358381    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcqwh\" (UniqueName: \"kubernetes.io/projected/fb36e980-e40a-42e8-b1db-f032b02d683a-kube-api-access-bcqwh\") pod \"ovnkube-node-sltmp\" (UID: \"fb36e980-e40a-42e8-b1db-f032b02d683a\") " pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.431775 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.431719    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wc65r"
Jul 16 15:35:36.438881 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.438863    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xv6dq"
Jul 16 15:35:36.442482 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:36.442459    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be4b188_7ce2_4eba_8936_425c93d1fdad.slice/crio-22c32a3e154d94d5460ba229ddb5bafe757e3b88ae44198598416ab08a5fbfe9 WatchSource:0}: Error finding container 22c32a3e154d94d5460ba229ddb5bafe757e3b88ae44198598416ab08a5fbfe9: Status 404 returned error can't find the container with id 22c32a3e154d94d5460ba229ddb5bafe757e3b88ae44198598416ab08a5fbfe9
Jul 16 15:35:36.446411 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.446391    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69"
Jul 16 15:35:36.448903 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:36.448884    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1a2a201_9a63_40aa_bc2c_84b68a6a6a24.slice/crio-c19ad559e365555ab657abe9b06b6442c1c83a7f2cdac59f0d6ac32a4b66e4b9 WatchSource:0}: Error finding container c19ad559e365555ab657abe9b06b6442c1c83a7f2cdac59f0d6ac32a4b66e4b9: Status 404 returned error can't find the container with id c19ad559e365555ab657abe9b06b6442c1c83a7f2cdac59f0d6ac32a4b66e4b9
Jul 16 15:35:36.449544 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.449522    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ftrpc"
Jul 16 15:35:36.456949 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.456908    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vk7rr"
Jul 16 15:35:36.457519 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:36.457496    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ee235c_ac9d_418f_a4b5_cab77e7f5404.slice/crio-5c26e759e0ae7074417a6306e2a8c44ac866abf33d60017c997c12f93908ac33 WatchSource:0}: Error finding container 5c26e759e0ae7074417a6306e2a8c44ac866abf33d60017c997c12f93908ac33: Status 404 returned error can't find the container with id 5c26e759e0ae7074417a6306e2a8c44ac866abf33d60017c997c12f93908ac33
Jul 16 15:35:36.459287 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:36.459266    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod764123aa_64ce_40d5_9d5f_8be36c2026f6.slice/crio-9e3bd1abbc54b3635998bb5562e58565a3676ea53f823117bf1906a344ebc643 WatchSource:0}: Error finding container 9e3bd1abbc54b3635998bb5562e58565a3676ea53f823117bf1906a344ebc643: Status 404 returned error can't find the container with id 9e3bd1abbc54b3635998bb5562e58565a3676ea53f823117bf1906a344ebc643
Jul 16 15:35:36.462222 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.461920    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:35:36.467139 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.467099    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b4cff"
Jul 16 15:35:36.467233 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:36.467208    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd00c5a27_627f_40bc_9ca7_48b091ff51c9.slice/crio-9101c18b0bf70420f754ce394e7c89bc0198ac071718787684233c360f3e485f WatchSource:0}: Error finding container 9101c18b0bf70420f754ce394e7c89bc0198ac071718787684233c360f3e485f: Status 404 returned error can't find the container with id 9101c18b0bf70420f754ce394e7c89bc0198ac071718787684233c360f3e485f
Jul 16 15:35:36.472361 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:36.472330    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb36e980_e40a_42e8_b1db_f032b02d683a.slice/crio-99e856297a90596a9d9322299d9143e39f19763bde5ec482b74f8c9a2a8b9903 WatchSource:0}: Error finding container 99e856297a90596a9d9322299d9143e39f19763bde5ec482b74f8c9a2a8b9903: Status 404 returned error can't find the container with id 99e856297a90596a9d9322299d9143e39f19763bde5ec482b74f8c9a2a8b9903
Jul 16 15:35:36.472425 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.472398    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-49wgv"
Jul 16 15:35:36.477690 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.476879    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kqh5g"
Jul 16 15:35:36.480226 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:36.480205    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51bd5dc0_5c9d_4c3f_b95b_1e275a3f8730.slice/crio-1ec68b82367d5a842087982b71ac52643441a60f67a84ffb0c1f56f55abb158e WatchSource:0}: Error finding container 1ec68b82367d5a842087982b71ac52643441a60f67a84ffb0c1f56f55abb158e: Status 404 returned error can't find the container with id 1ec68b82367d5a842087982b71ac52643441a60f67a84ffb0c1f56f55abb158e
Jul 16 15:35:36.486358 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:36.486336    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeea902f9_fb25_4b75_94ce_bce7bd7a41d9.slice/crio-30b494dd997ee69e20885cd4b48c8d363130a3d2bb78f54bba54f9a747e748eb WatchSource:0}: Error finding container 30b494dd997ee69e20885cd4b48c8d363130a3d2bb78f54bba54f9a747e748eb: Status 404 returned error can't find the container with id 30b494dd997ee69e20885cd4b48c8d363130a3d2bb78f54bba54f9a747e748eb
Jul 16 15:35:36.489318 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:35:36.489297    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a32f024_c920_4db1_8c81_37477f8dc910.slice/crio-5314fc136696a4420ecc2b0d00d18cb4586446a696e3baf10d8d627d25d07571 WatchSource:0}: Error finding container 5314fc136696a4420ecc2b0d00d18cb4586446a696e3baf10d8d627d25d07571: Status 404 returned error can't find the container with id 5314fc136696a4420ecc2b0d00d18cb4586446a696e3baf10d8d627d25d07571
Jul 16 15:35:36.748380 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.748302    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs\") pod \"network-metrics-daemon-98ggn\" (UID: \"52710c6b-b4c4-48db-9637-2f94d8c959bb\") " pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:36.748519 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.748489    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 15:35:36.748574 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.748562    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs podName:52710c6b-b4c4-48db-9637-2f94d8c959bb nodeName:}" failed. No retries permitted until 2026-07-16 15:35:37.748540852 +0000 UTC m=+3.937404799 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs") pod "network-metrics-daemon-98ggn" (UID: "52710c6b-b4c4-48db-9637-2f94d8c959bb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 15:35:36.849508 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.849475    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:36.849681 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.849565    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 15:35:36.849681 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.849627    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret podName:83adbeaf-0de4-4fec-be65-5a9a8b9ce326 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:37.849607741 +0000 UTC m=+4.038471674 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret") pod "global-pull-secret-syncer-4t8xm" (UID: "83adbeaf-0de4-4fec-be65-5a9a8b9ce326") : object "kube-system"/"original-pull-secret" not registered
Jul 16 15:35:36.950156 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:36.950123    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2fnt\" (UniqueName: \"kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt\") pod \"network-check-target-8jdhr\" (UID: \"958875c8-4f2f-474f-8201-b9090b15b099\") " pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:36.950328 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.950310    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 15:35:36.950385 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.950334    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 15:35:36.950385 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.950347    2570 projected.go:196] Error preparing data for projected volume kube-api-access-n2fnt for pod openshift-network-diagnostics/network-check-target-8jdhr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 15:35:36.950505 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:36.950410    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt podName:958875c8-4f2f-474f-8201-b9090b15b099 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:37.95038973 +0000 UTC m=+4.139253683 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2fnt" (UniqueName: "kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt") pod "network-check-target-8jdhr" (UID: "958875c8-4f2f-474f-8201-b9090b15b099") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 15:35:37.115336 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.115305    2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 15:35:37.197317 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.197237    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 15:30:36 +0000 UTC" deadline="2028-03-06 05:25:36.63091709 +0000 UTC"
Jul 16 15:35:37.197317 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.197296    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14365h49m59.433624903s"
Jul 16 15:35:37.261828 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.261802    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:37.261972 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:37.261952    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:35:37.271848 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.271620    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xv6dq" event={"ID":"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24","Type":"ContainerStarted","Data":"c19ad559e365555ab657abe9b06b6442c1c83a7f2cdac59f0d6ac32a4b66e4b9"}
Jul 16 15:35:37.275235 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.275038    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-49wgv" event={"ID":"eea902f9-fb25-4b75-94ce-bce7bd7a41d9","Type":"ContainerStarted","Data":"30b494dd997ee69e20885cd4b48c8d363130a3d2bb78f54bba54f9a747e748eb"}
Jul 16 15:35:37.282756 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.282733    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b4cff" event={"ID":"51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730","Type":"ContainerStarted","Data":"1ec68b82367d5a842087982b71ac52643441a60f67a84ffb0c1f56f55abb158e"}
Jul 16 15:35:37.287300 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.287275    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vk7rr" event={"ID":"d00c5a27-627f-40bc-9ca7-48b091ff51c9","Type":"ContainerStarted","Data":"9101c18b0bf70420f754ce394e7c89bc0198ac071718787684233c360f3e485f"}
Jul 16 15:35:37.291853 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.291796    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc65r" event={"ID":"7be4b188-7ce2-4eba-8936-425c93d1fdad","Type":"ContainerStarted","Data":"22c32a3e154d94d5460ba229ddb5bafe757e3b88ae44198598416ab08a5fbfe9"}
Jul 16 15:35:37.293828 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.293804    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-84.ec2.internal" event={"ID":"391d2eed45bf7533effc099da20934d8","Type":"ContainerStarted","Data":"605bb9980db064a5dc15b022929f79eddd908c1fe22ef0dfb0734a8b60266833"}
Jul 16 15:35:37.296822 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.296747    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kqh5g" event={"ID":"6a32f024-c920-4db1-8c81-37477f8dc910","Type":"ContainerStarted","Data":"5314fc136696a4420ecc2b0d00d18cb4586446a696e3baf10d8d627d25d07571"}
Jul 16 15:35:37.299643 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.299603    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp" event={"ID":"fb36e980-e40a-42e8-b1db-f032b02d683a","Type":"ContainerStarted","Data":"99e856297a90596a9d9322299d9143e39f19763bde5ec482b74f8c9a2a8b9903"}
Jul 16 15:35:37.304049 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.304024    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ftrpc" event={"ID":"764123aa-64ce-40d5-9d5f-8be36c2026f6","Type":"ContainerStarted","Data":"9e3bd1abbc54b3635998bb5562e58565a3676ea53f823117bf1906a344ebc643"}
Jul 16 15:35:37.314955 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.314909    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69" event={"ID":"71ee235c-ac9d-418f-a4b5-cab77e7f5404","Type":"ContainerStarted","Data":"5c26e759e0ae7074417a6306e2a8c44ac866abf33d60017c997c12f93908ac33"}
Jul 16 15:35:37.758306 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.758184    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs\") pod \"network-metrics-daemon-98ggn\" (UID: \"52710c6b-b4c4-48db-9637-2f94d8c959bb\") " pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:37.758466 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:37.758373    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 15:35:37.758466 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:37.758438    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs podName:52710c6b-b4c4-48db-9637-2f94d8c959bb nodeName:}" failed. No retries permitted until 2026-07-16 15:35:39.758420405 +0000 UTC m=+5.947284340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs") pod "network-metrics-daemon-98ggn" (UID: "52710c6b-b4c4-48db-9637-2f94d8c959bb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 15:35:37.858924 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.858890    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:37.859107 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:37.859041    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 15:35:37.859107 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:37.859099    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret podName:83adbeaf-0de4-4fec-be65-5a9a8b9ce326 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:39.859082699 +0000 UTC m=+6.047946632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret") pod "global-pull-secret-syncer-4t8xm" (UID: "83adbeaf-0de4-4fec-be65-5a9a8b9ce326") : object "kube-system"/"original-pull-secret" not registered
Jul 16 15:35:37.960291 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:37.960260    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2fnt\" (UniqueName: \"kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt\") pod \"network-check-target-8jdhr\" (UID: \"958875c8-4f2f-474f-8201-b9090b15b099\") " pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:37.960479 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:37.960460    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 15:35:37.960561 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:37.960486    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 15:35:37.960561 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:37.960499    2570 projected.go:196] Error preparing data for projected volume kube-api-access-n2fnt for pod openshift-network-diagnostics/network-check-target-8jdhr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 15:35:37.960561 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:37.960553    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt podName:958875c8-4f2f-474f-8201-b9090b15b099 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:39.960536201 +0000 UTC m=+6.149400135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2fnt" (UniqueName: "kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt") pod "network-check-target-8jdhr" (UID: "958875c8-4f2f-474f-8201-b9090b15b099") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 15:35:38.198225 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:38.198146    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 15:30:36 +0000 UTC" deadline="2027-12-25 06:58:20.39728929 +0000 UTC"
Jul 16 15:35:38.198225 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:38.198183    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12639h22m42.199110449s"
Jul 16 15:35:38.267196 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:38.267151    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:38.267383 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:38.267315    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:35:38.267786 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:38.267766    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:38.267900 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:38.267874    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:35:38.329632 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:38.329598    2570 generic.go:356] "Generic (PLEG): container finished" podID="445e0b74d7f76eb893ad5f0460043313" containerID="136966c072cd05eb9442b93baebb4116d0f7d39a97ece1ad1324343de341b0bd" exitCode=0
Jul 16 15:35:38.329791 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:38.329645    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal" event={"ID":"445e0b74d7f76eb893ad5f0460043313","Type":"ContainerDied","Data":"136966c072cd05eb9442b93baebb4116d0f7d39a97ece1ad1324343de341b0bd"}
Jul 16 15:35:39.262425 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:39.261955    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:39.262425 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:39.262084    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:35:39.343206 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:39.343172    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal" event={"ID":"445e0b74d7f76eb893ad5f0460043313","Type":"ContainerStarted","Data":"cc67cb7e51d357943eb9374706d21fc614a91dbf057e219900192cab6f581b45"}
Jul 16 15:35:39.359768 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:39.359713    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-84.ec2.internal" podStartSLOduration=4.359693174 podStartE2EDuration="4.359693174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:39.358868104 +0000 UTC m=+5.547732060" watchObservedRunningTime="2026-07-16 15:35:39.359693174 +0000 UTC m=+5.548557130"
Jul 16 15:35:39.779005 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:39.778965    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs\") pod \"network-metrics-daemon-98ggn\" (UID: \"52710c6b-b4c4-48db-9637-2f94d8c959bb\") " pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:39.779176 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:39.779105    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 15:35:39.779176 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:39.779160    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs podName:52710c6b-b4c4-48db-9637-2f94d8c959bb nodeName:}" failed. No retries permitted until 2026-07-16 15:35:43.779142775 +0000 UTC m=+9.968006715 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs") pod "network-metrics-daemon-98ggn" (UID: "52710c6b-b4c4-48db-9637-2f94d8c959bb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 15:35:39.880148 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:39.880114    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:39.880305 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:39.880269    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 15:35:39.880379 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:39.880336    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret podName:83adbeaf-0de4-4fec-be65-5a9a8b9ce326 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:43.88031713 +0000 UTC m=+10.069181067 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret") pod "global-pull-secret-syncer-4t8xm" (UID: "83adbeaf-0de4-4fec-be65-5a9a8b9ce326") : object "kube-system"/"original-pull-secret" not registered
Jul 16 15:35:39.980692 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:39.980656    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2fnt\" (UniqueName: \"kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt\") pod \"network-check-target-8jdhr\" (UID: \"958875c8-4f2f-474f-8201-b9090b15b099\") " pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:39.980874 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:39.980842    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 15:35:39.980874 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:39.980865    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 15:35:39.980874 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:39.980877    2570 projected.go:196] Error preparing data for projected volume kube-api-access-n2fnt for pod openshift-network-diagnostics/network-check-target-8jdhr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 15:35:39.981150 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:39.980948    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt podName:958875c8-4f2f-474f-8201-b9090b15b099 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:43.980913398 +0000 UTC m=+10.169777336 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2fnt" (UniqueName: "kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt") pod "network-check-target-8jdhr" (UID: "958875c8-4f2f-474f-8201-b9090b15b099") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 15:35:40.262238 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:40.262048    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:40.262238 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:40.262084    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:40.262238 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:40.262176    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:35:40.262815 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:40.262627    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:35:41.261668 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:41.261633    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:41.261854 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:41.261782    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:35:42.261708 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:42.261635    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:42.262156 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:42.261766    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:35:42.262281 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:42.262256    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:42.262421 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:42.262381    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:35:43.261960 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:43.261608    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:43.261960 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:43.261749    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:35:43.814752 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:43.814715    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs\") pod \"network-metrics-daemon-98ggn\" (UID: \"52710c6b-b4c4-48db-9637-2f94d8c959bb\") " pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:43.814962 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:43.814918    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 15:35:43.815033 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:43.814993    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs podName:52710c6b-b4c4-48db-9637-2f94d8c959bb nodeName:}" failed. No retries permitted until 2026-07-16 15:35:51.814975965 +0000 UTC m=+18.003839900 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs") pod "network-metrics-daemon-98ggn" (UID: "52710c6b-b4c4-48db-9637-2f94d8c959bb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 15:35:43.915906 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:43.915810    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:43.916093 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:43.915983    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 15:35:43.916093 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:43.916046    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret podName:83adbeaf-0de4-4fec-be65-5a9a8b9ce326 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:51.916028234 +0000 UTC m=+18.104892182 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret") pod "global-pull-secret-syncer-4t8xm" (UID: "83adbeaf-0de4-4fec-be65-5a9a8b9ce326") : object "kube-system"/"original-pull-secret" not registered
Jul 16 15:35:44.017163 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:44.017121    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2fnt\" (UniqueName: \"kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt\") pod \"network-check-target-8jdhr\" (UID: \"958875c8-4f2f-474f-8201-b9090b15b099\") " pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:44.017341 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:44.017293    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 15:35:44.017341 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:44.017317    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 15:35:44.017341 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:44.017329    2570 projected.go:196] Error preparing data for projected volume kube-api-access-n2fnt for pod openshift-network-diagnostics/network-check-target-8jdhr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 15:35:44.017504 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:44.017388    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt podName:958875c8-4f2f-474f-8201-b9090b15b099 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:52.017369405 +0000 UTC m=+18.206233340 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2fnt" (UniqueName: "kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt") pod "network-check-target-8jdhr" (UID: "958875c8-4f2f-474f-8201-b9090b15b099") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 15:35:44.262471 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:44.262016    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:44.262471 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:44.262121    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:35:44.262471 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:44.262292    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:44.262471 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:44.262386    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:35:45.261883 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:45.261847    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:45.262083 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:45.262022    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:35:46.261750 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:46.261565    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:46.261750 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:46.261704    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:35:46.262349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:46.261755    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:46.262349 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:46.261859    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:35:47.261200 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:47.261167    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:47.261338 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:47.261299    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:35:48.261372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:48.261336    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:48.261799 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:48.261473    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:35:48.261799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:48.261536    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:48.261799 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:48.261642    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:35:49.261491 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:49.261461    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:49.261918 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:49.261578    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:35:50.261370 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:50.261340    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:50.261518 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:50.261437    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:35:50.261518 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:50.261496    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:50.261887 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:50.261595    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:35:51.261332 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:51.261293    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:51.261545 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:51.261433    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:35:51.876156 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:51.876120    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs\") pod \"network-metrics-daemon-98ggn\" (UID: \"52710c6b-b4c4-48db-9637-2f94d8c959bb\") " pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:51.876340 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:51.876259    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 15:35:51.876340 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:51.876330    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs podName:52710c6b-b4c4-48db-9637-2f94d8c959bb nodeName:}" failed. No retries permitted until 2026-07-16 15:36:07.876307978 +0000 UTC m=+34.065171929 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs") pod "network-metrics-daemon-98ggn" (UID: "52710c6b-b4c4-48db-9637-2f94d8c959bb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 15:35:51.976990 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:51.976955    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:51.977147 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:51.977102    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 15:35:51.977206 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:51.977162    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret podName:83adbeaf-0de4-4fec-be65-5a9a8b9ce326 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:07.977143525 +0000 UTC m=+34.166007467 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret") pod "global-pull-secret-syncer-4t8xm" (UID: "83adbeaf-0de4-4fec-be65-5a9a8b9ce326") : object "kube-system"/"original-pull-secret" not registered
Jul 16 15:35:52.078057 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:52.078024    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2fnt\" (UniqueName: \"kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt\") pod \"network-check-target-8jdhr\" (UID: \"958875c8-4f2f-474f-8201-b9090b15b099\") " pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:52.078220 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:52.078192    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 15:35:52.078220 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:52.078219    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 15:35:52.078340 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:52.078233    2570 projected.go:196] Error preparing data for projected volume kube-api-access-n2fnt for pod openshift-network-diagnostics/network-check-target-8jdhr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 15:35:52.078340 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:52.078296    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt podName:958875c8-4f2f-474f-8201-b9090b15b099 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:08.078278764 +0000 UTC m=+34.267142718 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2fnt" (UniqueName: "kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt") pod "network-check-target-8jdhr" (UID: "958875c8-4f2f-474f-8201-b9090b15b099") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 15:35:52.261946 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:52.261856    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:52.262363 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:52.261858    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:52.262363 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:52.261996    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:35:52.262363 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:52.262058    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:35:53.261535 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:53.261498    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:53.261690 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:53.261621    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:35:54.262027 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:54.261996    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:54.262469 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:54.262108    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:35:54.262469 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:54.262162    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:54.262469 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:54.262282    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:35:55.261324 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:55.261290    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:55.261474 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:55.261401    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:35:56.260970 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:56.260944    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:56.261271 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:56.260951    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:56.261271 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:56.261070    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:35:56.261271 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:56.261100    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:35:56.378657 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:56.378616    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-84.ec2.internal" event={"ID":"391d2eed45bf7533effc099da20934d8","Type":"ContainerStarted","Data":"73f6985dcd000d6e89c9ee56ead8c855bea656b6f1800b0adfa01101c01b35c8"}
Jul 16 15:35:56.406765 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:56.406511    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-84.ec2.internal" podStartSLOduration=21.406491071 podStartE2EDuration="21.406491071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:56.405531298 +0000 UTC m=+22.594395253" watchObservedRunningTime="2026-07-16 15:35:56.406491071 +0000 UTC m=+22.595355027"
Jul 16 15:35:57.261397 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.261242    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:57.262047 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:57.261474    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:35:57.381264 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.381232    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-49wgv" event={"ID":"eea902f9-fb25-4b75-94ce-bce7bd7a41d9","Type":"ContainerStarted","Data":"e11bfba5b3b493a3fc087bd2f9ea7cdc2f0816290a0259254b53604741cdf5ae"}
Jul 16 15:35:57.382631 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.382609    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b4cff" event={"ID":"51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730","Type":"ContainerStarted","Data":"f21c2c7689bd5d40b5f9461e854f132378f193dae0b9553eaef38318e513481e"}
Jul 16 15:35:57.383902 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.383876    2570 generic.go:356] "Generic (PLEG): container finished" podID="7be4b188-7ce2-4eba-8936-425c93d1fdad" containerID="72a818d784c0d2141e425d250d3c8682c376133476ffd4c2f4a4c2d04da710da" exitCode=0
Jul 16 15:35:57.383999 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.383912    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc65r" event={"ID":"7be4b188-7ce2-4eba-8936-425c93d1fdad","Type":"ContainerDied","Data":"72a818d784c0d2141e425d250d3c8682c376133476ffd4c2f4a4c2d04da710da"}
Jul 16 15:35:57.385482 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.385456    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kqh5g" event={"ID":"6a32f024-c920-4db1-8c81-37477f8dc910","Type":"ContainerStarted","Data":"cc76821160a18027a9f36765ba903167961af2c419269ff7c159fb71014b45ea"}
Jul 16 15:35:57.388481 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.388452    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 15:35:57.391047 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.391024    2570 generic.go:356] "Generic (PLEG): container finished" podID="fb36e980-e40a-42e8-b1db-f032b02d683a" containerID="c6b81db672771d7a36c336f04b3c63f7861951c7e2fbff29f4493658d7163492" exitCode=1
Jul 16 15:35:57.391136 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.391088    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp" event={"ID":"fb36e980-e40a-42e8-b1db-f032b02d683a","Type":"ContainerStarted","Data":"cd0b80f7aa3416bc03d685744f0ff573a0c658b61c7d84af70c3a187775186ab"}
Jul 16 15:35:57.391189 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.391176    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp" event={"ID":"fb36e980-e40a-42e8-b1db-f032b02d683a","Type":"ContainerStarted","Data":"23d6f511487d92b1a42d40c1f321923234238bfcb433ed9afaea10e5a6d45725"}
Jul 16 15:35:57.391241 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.391217    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp" event={"ID":"fb36e980-e40a-42e8-b1db-f032b02d683a","Type":"ContainerStarted","Data":"f3c4dccd7cadcb118a515bb571a3a2d76975af3d7efb5c9fbdffc275842f29d9"}
Jul 16 15:35:57.391286 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.391245    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp" event={"ID":"fb36e980-e40a-42e8-b1db-f032b02d683a","Type":"ContainerStarted","Data":"f1a610d9fe22048545fe06e4fbb39d8ca8c515e784c0779670b857d27ae5a63f"}
Jul 16 15:35:57.391286 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.391271    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp" event={"ID":"fb36e980-e40a-42e8-b1db-f032b02d683a","Type":"ContainerDied","Data":"c6b81db672771d7a36c336f04b3c63f7861951c7e2fbff29f4493658d7163492"}
Jul 16 15:35:57.391399 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.391309    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp" event={"ID":"fb36e980-e40a-42e8-b1db-f032b02d683a","Type":"ContainerStarted","Data":"d0164c77f9970cf3c61ab30a615adedcff20278278b922285deeaf74fd9ed54a"}
Jul 16 15:35:57.392396 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.392375    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ftrpc" event={"ID":"764123aa-64ce-40d5-9d5f-8be36c2026f6","Type":"ContainerStarted","Data":"49458b37b1432702c6e748b7d9e77d1ed0ca3829c6d9ba55aecddbcd2822283f"}
Jul 16 15:35:57.393693 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.393675    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69" event={"ID":"71ee235c-ac9d-418f-a4b5-cab77e7f5404","Type":"ContainerStarted","Data":"781bf9a61678e5ad18199990cc033a4b36e1a55c3a48c8147e26174b1948621d"}
Jul 16 15:35:57.394888 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.394870    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xv6dq" event={"ID":"c1a2a201-9a63-40aa-bc2c-84b68a6a6a24","Type":"ContainerStarted","Data":"e8c72beb23787ba4cea2908569312f56965ce0604b1200b5946b7524ed0d97bf"}
Jul 16 15:35:57.400171 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.400136    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-49wgv" podStartSLOduration=3.642923639 podStartE2EDuration="23.400123825s" totalImagesPullingTime="19.757200186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:57.399926397 +0000 UTC m=+23.588790352" watchObservedRunningTime="2026-07-16 15:35:57.400123825 +0000 UTC m=+23.588987780"
Jul 16 15:35:57.415451 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.415417    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kqh5g" podStartSLOduration=3.6610213419999997 podStartE2EDuration="23.415406778s" totalImagesPullingTime="19.754385436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:57.415268246 +0000 UTC m=+23.604132201" watchObservedRunningTime="2026-07-16 15:35:57.415406778 +0000 UTC m=+23.604270733"
Jul 16 15:35:57.433195 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.433165    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xv6dq" podStartSLOduration=3.1976625690000002 podStartE2EDuration="23.433151522s" totalImagesPullingTime="20.235488953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:57.432830185 +0000 UTC m=+23.621694141" watchObservedRunningTime="2026-07-16 15:35:57.433151522 +0000 UTC m=+23.622015476"
Jul 16 15:35:57.446648 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.446612    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ftrpc" podStartSLOduration=3.665557571 podStartE2EDuration="23.446603375s" totalImagesPullingTime="19.781045804s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:57.446324644 +0000 UTC m=+23.635188598" watchObservedRunningTime="2026-07-16 15:35:57.446603375 +0000 UTC m=+23.635467330"
Jul 16 15:35:57.461146 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.461118    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b4cff" podStartSLOduration=3.7325083230000002 podStartE2EDuration="23.461108441s" totalImagesPullingTime="19.728600118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:57.461038646 +0000 UTC m=+23.649902614" watchObservedRunningTime="2026-07-16 15:35:57.461108441 +0000 UTC m=+23.649972379"
Jul 16 15:35:57.588323 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.588303    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:35:57.745346 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:57.745214    2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 16 15:35:58.190389 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.190352    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:35:58.204243 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.204159    2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T15:35:57.745343621Z","UUID":"e8ca2c85-c9a7-407d-a2c1-30b9ce1bcb74","Handler":null,"Name":"","Endpoint":""}
Jul 16 15:35:58.205731 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.205715    2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 16 15:35:58.205830 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.205758    2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 16 15:35:58.244100 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.244068    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b4cff"
Jul 16 15:35:58.261450 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.261432    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:35:58.261813 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.261432    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:35:58.261813 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:58.261542    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:35:58.261813 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:58.261621    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:35:58.391712 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.391674    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:35:58.399180 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.399151    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69" event={"ID":"71ee235c-ac9d-418f-a4b5-cab77e7f5404","Type":"ContainerStarted","Data":"600283ab92fe411a68886015b51bc072518ddad07ecc786882dc1f70e2b0f15c"}
Jul 16 15:35:58.400998 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.400967    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vk7rr" event={"ID":"d00c5a27-627f-40bc-9ca7-48b091ff51c9","Type":"ContainerStarted","Data":"7630511fbeb909dbbd78086018b699a17b635861ca6ada692ea7d8478fb07082"}
Jul 16 15:35:58.420378 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.420333    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vk7rr" podStartSLOduration=4.678625644 podStartE2EDuration="24.420314148s" totalImagesPullingTime="19.741688504s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:58.419983243 +0000 UTC m=+24.608847219" watchObservedRunningTime="2026-07-16 15:35:58.420314148 +0000 UTC m=+24.609178106"
Jul 16 15:35:58.450600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.450525    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:35:58.664566 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.664531    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:35:58.723526 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.723453    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:35:58.900595 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.900560    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:35:58.993070 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:58.992970    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:35:59.261639 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:59.261556    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:35:59.262158 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:35:59.261691    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:35:59.406188 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:59.406169    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 15:35:59.406597 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:35:59.406567    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp" event={"ID":"fb36e980-e40a-42e8-b1db-f032b02d683a","Type":"ContainerStarted","Data":"1110227cff05f6a4e0b4fde2b62fc9f1fefc45d58585ea1d4d3b59a7e4145a1c"}
Jul 16 15:36:00.227235 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:00.227198    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:00.261246 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:00.261216    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:36:00.261407 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:00.261343    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:36:00.261623 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:00.261216    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:36:00.261623 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:00.261585    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:36:00.326332 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:00.326300    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:00.411309 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:00.411275    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69" event={"ID":"71ee235c-ac9d-418f-a4b5-cab77e7f5404","Type":"ContainerStarted","Data":"7940254ac4a327787e26916e4db67bb20a8ad7fd79e9a4492e128816665bd036"}
Jul 16 15:36:00.431010 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:00.430968    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sh69" podStartSLOduration=3.610687246 podStartE2EDuration="26.430954162s" totalImagesPullingTime="22.820266916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:00.430486031 +0000 UTC m=+26.619350007" watchObservedRunningTime="2026-07-16 15:36:00.430954162 +0000 UTC m=+26.619818116"
Jul 16 15:36:01.261260 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:01.261222    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:36:01.261544 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:01.261367    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:36:02.102815 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.102588    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b4cff"
Jul 16 15:36:02.103262 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.103211    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b4cff"
Jul 16 15:36:02.261512 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.261434    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:36:02.261668 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:02.261532    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:36:02.261668 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.261621    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:36:02.261769 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:02.261700    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:36:02.417232 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.417200    2570 generic.go:356] "Generic (PLEG): container finished" podID="7be4b188-7ce2-4eba-8936-425c93d1fdad" containerID="e066c68666e3473f457232b9c20d8631b0262b2f39b098b7dd9f35522caae983" exitCode=0
Jul 16 15:36:02.417406 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.417283    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc65r" event={"ID":"7be4b188-7ce2-4eba-8936-425c93d1fdad","Type":"ContainerDied","Data":"e066c68666e3473f457232b9c20d8631b0262b2f39b098b7dd9f35522caae983"}
Jul 16 15:36:02.420775 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.420753    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 15:36:02.421187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.421163    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp" event={"ID":"fb36e980-e40a-42e8-b1db-f032b02d683a","Type":"ContainerStarted","Data":"9171e5061f3365aec9717e546b4b9a3d8fade1e35b202d45b75bfe045674c86b"}
Jul 16 15:36:02.421449 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.421383    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:36:02.421542 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.421471    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:36:02.421542 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.421522    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:36:02.422440 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.421570    2570 scope.go:117] "RemoveContainer" containerID="c6b81db672771d7a36c336f04b3c63f7861951c7e2fbff29f4493658d7163492"
Jul 16 15:36:02.422552 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.422490    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b4cff"
Jul 16 15:36:02.437964 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.437921    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:36:02.438049 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.438042    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:36:02.687925 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:02.687891    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:03.261390 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:03.261185    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:36:03.261842 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:03.261499    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:36:03.426712 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:03.426688    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 15:36:03.427077 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:03.427053    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp" event={"ID":"fb36e980-e40a-42e8-b1db-f032b02d683a","Type":"ContainerStarted","Data":"c7cd29ae19ac85c929e769b1873311019502b5c59f7e8498fc43b3b2194463c3"}
Jul 16 15:36:03.460073 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:03.457653    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp" podStartSLOduration=9.64474386 podStartE2EDuration="29.457633138s" totalImagesPullingTime="19.812889278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:03.455785162 +0000 UTC m=+29.644649117" watchObservedRunningTime="2026-07-16 15:36:03.457633138 +0000 UTC m=+29.646497091"
Jul 16 15:36:03.501374 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:03.501341    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-98ggn"]
Jul 16 15:36:03.501554 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:03.501534    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:36:03.501686 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:03.501660    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:36:03.502094 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:03.502078    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8jdhr"]
Jul 16 15:36:03.502164 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:03.502154    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:36:03.502246 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:03.502222    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:36:03.503801 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:03.503779    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4t8xm"]
Jul 16 15:36:03.503900 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:03.503871    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:36:03.504003 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:03.503982    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:36:05.261240 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:05.261216    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:36:05.261240 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:05.261237    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:36:05.261664 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:05.261255    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:36:05.261664 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:05.261326    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:36:05.261664 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:05.261389    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:36:05.261664 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:05.261467    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:36:05.433147 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:05.433119    2570 generic.go:356] "Generic (PLEG): container finished" podID="7be4b188-7ce2-4eba-8936-425c93d1fdad" containerID="0d7ebbb1a6502f56554ee85ef1bdccd1ec99b0b8133d24c559f27275d2eb8736" exitCode=0
Jul 16 15:36:05.433283 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:05.433155    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc65r" event={"ID":"7be4b188-7ce2-4eba-8936-425c93d1fdad","Type":"ContainerDied","Data":"0d7ebbb1a6502f56554ee85ef1bdccd1ec99b0b8133d24c559f27275d2eb8736"}
Jul 16 15:36:06.420639 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:06.420435    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:06.473233 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:06.473203    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:06.825414 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:06.825318    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:06.907579 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:06.907545    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:06.961301 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:06.961269    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:07.261802 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:07.261777    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:36:07.261895 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:07.261801    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:36:07.261895 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:07.261801    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:36:07.261895 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:07.261865    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:36:07.262038 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:07.261968    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:36:07.262078 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:07.262043    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:36:07.439325 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:07.439303    2570 generic.go:356] "Generic (PLEG): container finished" podID="7be4b188-7ce2-4eba-8936-425c93d1fdad" containerID="3efcdecdac43d491a332325848942957e43f61aeeda6a0880f965e5c13395313" exitCode=0
Jul 16 15:36:07.439673 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:07.439352    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc65r" event={"ID":"7be4b188-7ce2-4eba-8936-425c93d1fdad","Type":"ContainerDied","Data":"3efcdecdac43d491a332325848942957e43f61aeeda6a0880f965e5c13395313"}
Jul 16 15:36:07.575327 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:07.575297    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:07.670943 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:07.670903    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:07.900295 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:07.900261    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs\") pod \"network-metrics-daemon-98ggn\" (UID: \"52710c6b-b4c4-48db-9637-2f94d8c959bb\") " pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:36:07.900445 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:07.900377    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 15:36:07.900445 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:07.900434    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs podName:52710c6b-b4c4-48db-9637-2f94d8c959bb nodeName:}" failed. No retries permitted until 2026-07-16 15:36:39.900415171 +0000 UTC m=+66.089279108 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs") pod "network-metrics-daemon-98ggn" (UID: "52710c6b-b4c4-48db-9637-2f94d8c959bb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 15:36:07.969405 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:07.969379    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kqh5g_6a32f024-c920-4db1-8c81-37477f8dc910/dns-node-resolver/0.log"
Jul 16 15:36:07.999965 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:07.999919    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:08.000767 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:08.000746    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:36:08.000879 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:08.000862    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 15:36:08.000921 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:08.000916    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret podName:83adbeaf-0de4-4fec-be65-5a9a8b9ce326 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:40.0009036 +0000 UTC m=+66.189767546 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret") pod "global-pull-secret-syncer-4t8xm" (UID: "83adbeaf-0de4-4fec-be65-5a9a8b9ce326") : object "kube-system"/"original-pull-secret" not registered
Jul 16 15:36:08.101120 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:08.101093    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2fnt\" (UniqueName: \"kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt\") pod \"network-check-target-8jdhr\" (UID: \"958875c8-4f2f-474f-8201-b9090b15b099\") " pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:36:08.101309 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:08.101274    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 15:36:08.101309 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:08.101302    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 15:36:08.101459 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:08.101316    2570 projected.go:196] Error preparing data for projected volume kube-api-access-n2fnt for pod openshift-network-diagnostics/network-check-target-8jdhr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 15:36:08.101459 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:08.101379    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt podName:958875c8-4f2f-474f-8201-b9090b15b099 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:40.101361082 +0000 UTC m=+66.290225018 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2fnt" (UniqueName: "kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt") pod "network-check-target-8jdhr" (UID: "958875c8-4f2f-474f-8201-b9090b15b099") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 15:36:08.102111 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:08.102053    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:08.274481 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:08.274400    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:08.370706 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:08.370672    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:08.612599 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:08.612561    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:08.668395 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:08.668362    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:08.769656 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:08.769623    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:09.170066 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.170042    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ftrpc_764123aa-64ce-40d5-9d5f-8be36c2026f6/node-ca/0.log"
Jul 16 15:36:09.261553 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.261521    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:36:09.261689 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.261559    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:36:09.261735 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.261702    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:36:09.261735 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:09.261716    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t8xm" podUID="83adbeaf-0de4-4fec-be65-5a9a8b9ce326"
Jul 16 15:36:09.261823 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:09.261789    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8jdhr" podUID="958875c8-4f2f-474f-8201-b9090b15b099"
Jul 16 15:36:09.261884 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:09.261854    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98ggn" podUID="52710c6b-b4c4-48db-9637-2f94d8c959bb"
Jul 16 15:36:09.539319 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.539287    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-84.ec2.internal" event="NodeReady"
Jul 16 15:36:09.539492 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.539436    2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 16 15:36:09.598038 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.598004    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75d6fbc47f-rjqcs"]
Jul 16 15:36:09.602319 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.602290    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.605175 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.605075    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 16 15:36:09.605175 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.605124    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 16 15:36:09.605365 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.605129    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 16 15:36:09.605365 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.605282    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8gcr9\""
Jul 16 15:36:09.611819 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.611788    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 16 15:36:09.615662 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.615644    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-77d2v"]
Jul 16 15:36:09.619873 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.619856    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:09.622596 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.622573    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 16 15:36:09.623286 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.623265    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vskms\""
Jul 16 15:36:09.623802 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.623625    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 16 15:36:09.624668 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.624643    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9bzgd"]
Jul 16 15:36:09.628762 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.628745    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75d6fbc47f-rjqcs"]
Jul 16 15:36:09.628852 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.628811    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9bzgd"
Jul 16 15:36:09.631924 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.631894    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 16 15:36:09.631924 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.631942    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 16 15:36:09.632118 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.631946    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 16 15:36:09.632118 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.631894    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6gbrz\""
Jul 16 15:36:09.635458 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.635412    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-77d2v"]
Jul 16 15:36:09.642126 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.642106    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9bzgd"]
Jul 16 15:36:09.655102 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.655080    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-b9qf6"]
Jul 16 15:36:09.659172 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.659152    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:09.662148 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.661926    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 16 15:36:09.662148 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.662018    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 16 15:36:09.662148 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.662074    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:09.662148 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.662136    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zzsd4\""
Jul 16 15:36:09.662409 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.662244    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 16 15:36:09.662499 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.662479    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 16 15:36:09.670377 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.670354    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b9qf6"]
Jul 16 15:36:09.714741 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.714706    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:09.714876 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.714744    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/692cf320-ee61-4f05-aa65-da1598213514-image-registry-private-configuration\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.714876 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.714806    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/692cf320-ee61-4f05-aa65-da1598213514-registry-certificates\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.715012 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.714896    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/692cf320-ee61-4f05-aa65-da1598213514-ca-trust-extracted\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.715012 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.714964    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/83c60a71-04d9-4ee3-8431-b381d610783c-config-volume\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:09.715012 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.714991    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-279gn\" (UniqueName: \"kubernetes.io/projected/83c60a71-04d9-4ee3-8431-b381d610783c-kube-api-access-279gn\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:09.715156 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.715023    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/692cf320-ee61-4f05-aa65-da1598213514-trusted-ca\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.715156 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.715047    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhzmn\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-kube-api-access-lhzmn\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.715156 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.715068    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-bound-sa-token\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.715300 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.715155    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83c60a71-04d9-4ee3-8431-b381d610783c-tmp-dir\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:09.715300 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.715179    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.715300 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.715201    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/692cf320-ee61-4f05-aa65-da1598213514-installation-pull-secrets\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.716776 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.716744    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:09.776693 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.776660    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:09.816576 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.816504    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:09.816576 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.816545    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/692cf320-ee61-4f05-aa65-da1598213514-image-registry-private-configuration\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.816576 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.816572    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/692cf320-ee61-4f05-aa65-da1598213514-registry-certificates\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.816801 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.816600    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/692cf320-ee61-4f05-aa65-da1598213514-ca-trust-extracted\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.816801 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:09.816626    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 15:36:09.816801 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.816626    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/83c60a71-04d9-4ee3-8431-b381d610783c-config-volume\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:09.816801 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:09.816699    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls podName:83c60a71-04d9-4ee3-8431-b381d610783c nodeName:}" failed. No retries permitted until 2026-07-16 15:36:10.316675514 +0000 UTC m=+36.505539455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls") pod "dns-default-77d2v" (UID: "83c60a71-04d9-4ee3-8431-b381d610783c") : secret "dns-default-metrics-tls" not found
Jul 16 15:36:09.816801 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.816735    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-279gn\" (UniqueName: \"kubernetes.io/projected/83c60a71-04d9-4ee3-8431-b381d610783c-kube-api-access-279gn\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:09.816801 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.816763    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b7607d9f-e61a-4b41-9494-4622fb17f848-crio-socket\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:09.816801 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.816792    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/692cf320-ee61-4f05-aa65-da1598213514-trusted-ca\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.817171 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.816817    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhzmn\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-kube-api-access-lhzmn\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.817171 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.816857    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-bound-sa-token\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.817171 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.816882    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-664kr\" (UniqueName: \"kubernetes.io/projected/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-api-access-664kr\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:09.817171 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.816954    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:09.817171 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.816989    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7607d9f-e61a-4b41-9494-4622fb17f848-data-volume\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:09.817171 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.817054    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83c60a71-04d9-4ee3-8431-b381d610783c-tmp-dir\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:09.817171 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.817078    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6bl2\" (UniqueName: \"kubernetes.io/projected/56a2820e-a5d6-43e8-ad74-3fa061bd5071-kube-api-access-z6bl2\") pod \"ingress-canary-9bzgd\" (UID: \"56a2820e-a5d6-43e8-ad74-3fa061bd5071\") " pod="openshift-ingress-canary/ingress-canary-9bzgd"
Jul 16 15:36:09.817171 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.817100    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/692cf320-ee61-4f05-aa65-da1598213514-ca-trust-extracted\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.817529 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.817184    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.817529 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.817214    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/692cf320-ee61-4f05-aa65-da1598213514-installation-pull-secrets\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.817529 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.817245    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56a2820e-a5d6-43e8-ad74-3fa061bd5071-cert\") pod \"ingress-canary-9bzgd\" (UID: \"56a2820e-a5d6-43e8-ad74-3fa061bd5071\") " pod="openshift-ingress-canary/ingress-canary-9bzgd"
Jul 16 15:36:09.817529 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.817276    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:09.817529 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.817285    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/692cf320-ee61-4f05-aa65-da1598213514-registry-certificates\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.817529 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.817374    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83c60a71-04d9-4ee3-8431-b381d610783c-tmp-dir\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:09.817529 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:09.817479    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 15:36:09.817529 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:09.817498    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75d6fbc47f-rjqcs: secret "image-registry-tls" not found
Jul 16 15:36:09.817921 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:09.817569    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls podName:692cf320-ee61-4f05-aa65-da1598213514 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:10.317551072 +0000 UTC m=+36.506415005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls") pod "image-registry-75d6fbc47f-rjqcs" (UID: "692cf320-ee61-4f05-aa65-da1598213514") : secret "image-registry-tls" not found
Jul 16 15:36:09.817921 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.817648    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/83c60a71-04d9-4ee3-8431-b381d610783c-config-volume\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:09.818043 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.817973    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/692cf320-ee61-4f05-aa65-da1598213514-trusted-ca\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.821626 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.821597    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/692cf320-ee61-4f05-aa65-da1598213514-installation-pull-secrets\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.821626 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.821617    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/692cf320-ee61-4f05-aa65-da1598213514-image-registry-private-configuration\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.826198 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.826180    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhzmn\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-kube-api-access-lhzmn\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.826460 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.826437    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-bound-sa-token\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:09.826593 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.826553    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-279gn\" (UniqueName: \"kubernetes.io/projected/83c60a71-04d9-4ee3-8431-b381d610783c-kube-api-access-279gn\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:09.878396 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.878351    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:09.881744 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.881720    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-klvpc"]
Jul 16 15:36:09.890318 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.886972    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-klvpc"
Jul 16 15:36:09.890318 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.890200    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 16 15:36:09.890491 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.890426    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 16 15:36:09.890491 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.890432    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 16 15:36:09.890595 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.890563    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-xpj6t\""
Jul 16 15:36:09.890753 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.890729    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 16 15:36:09.893101 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.893080    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-klvpc"]
Jul 16 15:36:09.917884 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.917854    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b7607d9f-e61a-4b41-9494-4622fb17f848-crio-socket\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:09.917997 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.917922    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-664kr\" (UniqueName: \"kubernetes.io/projected/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-api-access-664kr\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:09.918051 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.918017    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:09.918110 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.918047    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7607d9f-e61a-4b41-9494-4622fb17f848-data-volume\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:09.918110 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.918092    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b7607d9f-e61a-4b41-9494-4622fb17f848-crio-socket\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:09.918220 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.918098    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6bl2\" (UniqueName: \"kubernetes.io/projected/56a2820e-a5d6-43e8-ad74-3fa061bd5071-kube-api-access-z6bl2\") pod \"ingress-canary-9bzgd\" (UID: \"56a2820e-a5d6-43e8-ad74-3fa061bd5071\") " pod="openshift-ingress-canary/ingress-canary-9bzgd"
Jul 16 15:36:09.918220 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:09.918152    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 15:36:09.918220 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.918178    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56a2820e-a5d6-43e8-ad74-3fa061bd5071-cert\") pod \"ingress-canary-9bzgd\" (UID: \"56a2820e-a5d6-43e8-ad74-3fa061bd5071\") " pod="openshift-ingress-canary/ingress-canary-9bzgd"
Jul 16 15:36:09.918220 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:09.918207    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls podName:b7607d9f-e61a-4b41-9494-4622fb17f848 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:10.418190101 +0000 UTC m=+36.607054074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b9qf6" (UID: "b7607d9f-e61a-4b41-9494-4622fb17f848") : secret "insights-runtime-extractor-tls" not found
Jul 16 15:36:09.918375 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.918223    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:09.918375 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:09.918356    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 15:36:09.918454 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.918408    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7607d9f-e61a-4b41-9494-4622fb17f848-data-volume\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:09.918454 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:09.918412    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm podName:b7607d9f-e61a-4b41-9494-4622fb17f848 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:10.418397464 +0000 UTC m=+36.607261416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b9qf6" (UID: "b7607d9f-e61a-4b41-9494-4622fb17f848") : configmap "kube-rbac-proxy" not found
Jul 16 15:36:09.921332 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.921309    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56a2820e-a5d6-43e8-ad74-3fa061bd5071-cert\") pod \"ingress-canary-9bzgd\" (UID: \"56a2820e-a5d6-43e8-ad74-3fa061bd5071\") " pod="openshift-ingress-canary/ingress-canary-9bzgd"
Jul 16 15:36:09.929299 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.929273    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6bl2\" (UniqueName: \"kubernetes.io/projected/56a2820e-a5d6-43e8-ad74-3fa061bd5071-kube-api-access-z6bl2\") pod \"ingress-canary-9bzgd\" (UID: \"56a2820e-a5d6-43e8-ad74-3fa061bd5071\") " pod="openshift-ingress-canary/ingress-canary-9bzgd"
Jul 16 15:36:09.929434 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.929414    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-664kr\" (UniqueName: \"kubernetes.io/projected/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-api-access-664kr\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:09.931706 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.931679    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:09.939400 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:09.939381    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9bzgd"
Jul 16 15:36:10.019150 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.019118    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a56abb16-1ac4-489e-b740-5cbd71e6d090-signing-cabundle\") pod \"service-ca-79f68b9f4f-klvpc\" (UID: \"a56abb16-1ac4-489e-b740-5cbd71e6d090\") " pod="openshift-service-ca/service-ca-79f68b9f4f-klvpc"
Jul 16 15:36:10.019299 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.019267    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a56abb16-1ac4-489e-b740-5cbd71e6d090-signing-key\") pod \"service-ca-79f68b9f4f-klvpc\" (UID: \"a56abb16-1ac4-489e-b740-5cbd71e6d090\") " pod="openshift-service-ca/service-ca-79f68b9f4f-klvpc"
Jul 16 15:36:10.019299 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.019293    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqkmn\" (UniqueName: \"kubernetes.io/projected/a56abb16-1ac4-489e-b740-5cbd71e6d090-kube-api-access-rqkmn\") pod \"service-ca-79f68b9f4f-klvpc\" (UID: \"a56abb16-1ac4-489e-b740-5cbd71e6d090\") " pod="openshift-service-ca/service-ca-79f68b9f4f-klvpc"
Jul 16 15:36:10.119664 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.119630    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a56abb16-1ac4-489e-b740-5cbd71e6d090-signing-key\") pod \"service-ca-79f68b9f4f-klvpc\" (UID: \"a56abb16-1ac4-489e-b740-5cbd71e6d090\") " pod="openshift-service-ca/service-ca-79f68b9f4f-klvpc"
Jul 16 15:36:10.119811 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.119670    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqkmn\" (UniqueName: \"kubernetes.io/projected/a56abb16-1ac4-489e-b740-5cbd71e6d090-kube-api-access-rqkmn\") pod \"service-ca-79f68b9f4f-klvpc\" (UID: \"a56abb16-1ac4-489e-b740-5cbd71e6d090\") " pod="openshift-service-ca/service-ca-79f68b9f4f-klvpc"
Jul 16 15:36:10.119811 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.119698    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a56abb16-1ac4-489e-b740-5cbd71e6d090-signing-cabundle\") pod \"service-ca-79f68b9f4f-klvpc\" (UID: \"a56abb16-1ac4-489e-b740-5cbd71e6d090\") " pod="openshift-service-ca/service-ca-79f68b9f4f-klvpc"
Jul 16 15:36:10.120379 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.120358    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a56abb16-1ac4-489e-b740-5cbd71e6d090-signing-cabundle\") pod \"service-ca-79f68b9f4f-klvpc\" (UID: \"a56abb16-1ac4-489e-b740-5cbd71e6d090\") " pod="openshift-service-ca/service-ca-79f68b9f4f-klvpc"
Jul 16 15:36:10.122323 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.122296    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a56abb16-1ac4-489e-b740-5cbd71e6d090-signing-key\") pod \"service-ca-79f68b9f4f-klvpc\" (UID: \"a56abb16-1ac4-489e-b740-5cbd71e6d090\") " pod="openshift-service-ca/service-ca-79f68b9f4f-klvpc"
Jul 16 15:36:10.129086 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.129066    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqkmn\" (UniqueName: \"kubernetes.io/projected/a56abb16-1ac4-489e-b740-5cbd71e6d090-kube-api-access-rqkmn\") pod \"service-ca-79f68b9f4f-klvpc\" (UID: \"a56abb16-1ac4-489e-b740-5cbd71e6d090\") " pod="openshift-service-ca/service-ca-79f68b9f4f-klvpc"
Jul 16 15:36:10.199043 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.199014    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-klvpc"
Jul 16 15:36:10.321606 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.321575    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:10.321771 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.321631    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:10.321771 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:10.321738    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 15:36:10.321771 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:10.321761    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75d6fbc47f-rjqcs: secret "image-registry-tls" not found
Jul 16 15:36:10.321771 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:10.321739    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 15:36:10.321957 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:10.321820    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls podName:83c60a71-04d9-4ee3-8431-b381d610783c nodeName:}" failed. No retries permitted until 2026-07-16 15:36:11.321801936 +0000 UTC m=+37.510665889 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls") pod "dns-default-77d2v" (UID: "83c60a71-04d9-4ee3-8431-b381d610783c") : secret "dns-default-metrics-tls" not found
Jul 16 15:36:10.321957 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:10.321837    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls podName:692cf320-ee61-4f05-aa65-da1598213514 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:11.321828298 +0000 UTC m=+37.510692237 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls") pod "image-registry-75d6fbc47f-rjqcs" (UID: "692cf320-ee61-4f05-aa65-da1598213514") : secret "image-registry-tls" not found
Jul 16 15:36:10.323728 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.323707    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:10.422756 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.422677    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:10.422906 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.422759    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:10.422906 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:10.422801    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 15:36:10.422906 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:10.422870    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm podName:b7607d9f-e61a-4b41-9494-4622fb17f848 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:11.422853756 +0000 UTC m=+37.611717700 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b9qf6" (UID: "b7607d9f-e61a-4b41-9494-4622fb17f848") : configmap "kube-rbac-proxy" not found
Jul 16 15:36:10.422906 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:10.422874    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 15:36:10.422906 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:10.422907    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls podName:b7607d9f-e61a-4b41-9494-4622fb17f848 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:11.422897671 +0000 UTC m=+37.611761606 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b9qf6" (UID: "b7607d9f-e61a-4b41-9494-4622fb17f848") : secret "insights-runtime-extractor-tls" not found
Jul 16 15:36:10.424773 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:10.424741    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:11.261605 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.261551    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:36:11.262043 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.261566    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:36:11.262777 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.261566    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:36:11.264802 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.264782    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 15:36:11.266202 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.266183    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 15:36:11.266320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.266191    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 15:36:11.267284 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.267264    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2scpx\""
Jul 16 15:36:11.267284 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.267279    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zfqhq\""
Jul 16 15:36:11.267511 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.267495    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 15:36:11.289954 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.289765    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:11.330787 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.330749    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:11.330896 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.330834    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:11.330976 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:11.330889    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 15:36:11.330976 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:11.330909    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75d6fbc47f-rjqcs: secret "image-registry-tls" not found
Jul 16 15:36:11.331078 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:11.330978    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 15:36:11.331078 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:11.330991    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls podName:692cf320-ee61-4f05-aa65-da1598213514 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:13.330969606 +0000 UTC m=+39.519833553 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls") pod "image-registry-75d6fbc47f-rjqcs" (UID: "692cf320-ee61-4f05-aa65-da1598213514") : secret "image-registry-tls" not found
Jul 16 15:36:11.331078 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:11.331034    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls podName:83c60a71-04d9-4ee3-8431-b381d610783c nodeName:}" failed. No retries permitted until 2026-07-16 15:36:13.331016894 +0000 UTC m=+39.519880842 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls") pod "dns-default-77d2v" (UID: "83c60a71-04d9-4ee3-8431-b381d610783c") : secret "dns-default-metrics-tls" not found
Jul 16 15:36:11.378254 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.378067    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9bzgd"]
Jul 16 15:36:11.381995 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:36:11.381968    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a2820e_a5d6_43e8_ad74_3fa061bd5071.slice/crio-f7ce857b3f07651b120c74ba2eb860cf5ff73b07b9a53eb61249a20601dd8bb9 WatchSource:0}: Error finding container f7ce857b3f07651b120c74ba2eb860cf5ff73b07b9a53eb61249a20601dd8bb9: Status 404 returned error can't find the container with id f7ce857b3f07651b120c74ba2eb860cf5ff73b07b9a53eb61249a20601dd8bb9
Jul 16 15:36:11.381995 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.381980    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-klvpc"]
Jul 16 15:36:11.385225 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:36:11.385202    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda56abb16_1ac4_489e_b740_5cbd71e6d090.slice/crio-f9b3956754b631c3cc06d4e65dd633a4c1aa2fd5a09bb7acdb9ba72ed970dc4f WatchSource:0}: Error finding container f9b3956754b631c3cc06d4e65dd633a4c1aa2fd5a09bb7acdb9ba72ed970dc4f: Status 404 returned error can't find the container with id f9b3956754b631c3cc06d4e65dd633a4c1aa2fd5a09bb7acdb9ba72ed970dc4f
Jul 16 15:36:11.431699 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.431670    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:11.431803 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.431742    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:11.431857 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:11.431827    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 15:36:11.431857 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:11.431838    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 15:36:11.431990 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:11.431880    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm podName:b7607d9f-e61a-4b41-9494-4622fb17f848 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:13.431863746 +0000 UTC m=+39.620727679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b9qf6" (UID: "b7607d9f-e61a-4b41-9494-4622fb17f848") : configmap "kube-rbac-proxy" not found
Jul 16 15:36:11.431990 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:11.431895    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls podName:b7607d9f-e61a-4b41-9494-4622fb17f848 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:13.431887552 +0000 UTC m=+39.620751484 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b9qf6" (UID: "b7607d9f-e61a-4b41-9494-4622fb17f848") : secret "insights-runtime-extractor-tls" not found
Jul 16 15:36:11.454974 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.454926    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc65r" event={"ID":"7be4b188-7ce2-4eba-8936-425c93d1fdad","Type":"ContainerStarted","Data":"3bdf70652fab546dd8c367eebb4d5bc28a55d0f6a8f14f5b8c28eb7554a0f79d"}
Jul 16 15:36:11.456087 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.456059    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-klvpc" event={"ID":"a56abb16-1ac4-489e-b740-5cbd71e6d090","Type":"ContainerStarted","Data":"f9b3956754b631c3cc06d4e65dd633a4c1aa2fd5a09bb7acdb9ba72ed970dc4f"}
Jul 16 15:36:11.457118 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:11.457094    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9bzgd" event={"ID":"56a2820e-a5d6-43e8-ad74-3fa061bd5071","Type":"ContainerStarted","Data":"f7ce857b3f07651b120c74ba2eb860cf5ff73b07b9a53eb61249a20601dd8bb9"}
Jul 16 15:36:12.463816 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:12.463696    2570 generic.go:356] "Generic (PLEG): container finished" podID="7be4b188-7ce2-4eba-8936-425c93d1fdad" containerID="3bdf70652fab546dd8c367eebb4d5bc28a55d0f6a8f14f5b8c28eb7554a0f79d" exitCode=0
Jul 16 15:36:12.463816 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:12.463765    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc65r" event={"ID":"7be4b188-7ce2-4eba-8936-425c93d1fdad","Type":"ContainerDied","Data":"3bdf70652fab546dd8c367eebb4d5bc28a55d0f6a8f14f5b8c28eb7554a0f79d"}
Jul 16 15:36:13.348153 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:13.348113    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:13.348308 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:13.348182    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:13.348308 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:13.348274    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 15:36:13.348308 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:13.348297    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75d6fbc47f-rjqcs: secret "image-registry-tls" not found
Jul 16 15:36:13.348461 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:13.348343    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 15:36:13.348461 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:13.348364    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls podName:692cf320-ee61-4f05-aa65-da1598213514 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:17.348342839 +0000 UTC m=+43.537206771 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls") pod "image-registry-75d6fbc47f-rjqcs" (UID: "692cf320-ee61-4f05-aa65-da1598213514") : secret "image-registry-tls" not found
Jul 16 15:36:13.348461 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:13.348422    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls podName:83c60a71-04d9-4ee3-8431-b381d610783c nodeName:}" failed. No retries permitted until 2026-07-16 15:36:17.348386527 +0000 UTC m=+43.537250474 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls") pod "dns-default-77d2v" (UID: "83c60a71-04d9-4ee3-8431-b381d610783c") : secret "dns-default-metrics-tls" not found
Jul 16 15:36:13.449059 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:13.449026    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:13.449237 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:13.449124    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:13.449237 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:13.449139    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 15:36:13.449237 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:13.449196    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls podName:b7607d9f-e61a-4b41-9494-4622fb17f848 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:17.449179796 +0000 UTC m=+43.638043730 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b9qf6" (UID: "b7607d9f-e61a-4b41-9494-4622fb17f848") : secret "insights-runtime-extractor-tls" not found
Jul 16 15:36:13.449237 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:13.449223    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 15:36:13.449455 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:13.449273    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm podName:b7607d9f-e61a-4b41-9494-4622fb17f848 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:17.449257691 +0000 UTC m=+43.638121624 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b9qf6" (UID: "b7607d9f-e61a-4b41-9494-4622fb17f848") : configmap "kube-rbac-proxy" not found
Jul 16 15:36:13.469778 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:13.469743    2570 generic.go:356] "Generic (PLEG): container finished" podID="7be4b188-7ce2-4eba-8936-425c93d1fdad" containerID="787046118a54f2a6c46a2af4ded2b48bd4b1e5c9adc68905a764643ae4e04048" exitCode=0
Jul 16 15:36:13.470457 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:13.469794    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc65r" event={"ID":"7be4b188-7ce2-4eba-8936-425c93d1fdad","Type":"ContainerDied","Data":"787046118a54f2a6c46a2af4ded2b48bd4b1e5c9adc68905a764643ae4e04048"}
Jul 16 15:36:13.617457 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:13.617389    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:14.317330 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:14.317293    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:15.476315 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:15.476277    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9bzgd" event={"ID":"56a2820e-a5d6-43e8-ad74-3fa061bd5071","Type":"ContainerStarted","Data":"46d0ccdddcc4e5c548babd2c3ebfc5c136e81d617217203e277ec4532fd182d0"}
Jul 16 15:36:15.479690 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:15.479658    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc65r" event={"ID":"7be4b188-7ce2-4eba-8936-425c93d1fdad","Type":"ContainerStarted","Data":"b277fd2ca4883765e61931dac539abaab08d75139840a988e559dceb9686ee26"}
Jul 16 15:36:15.481108 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:15.481086    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-klvpc" event={"ID":"a56abb16-1ac4-489e-b740-5cbd71e6d090","Type":"ContainerStarted","Data":"5e14731c8d1621613bdbcc74fbc9c967bca788d743f23ec17dfc493682b57b1c"}
Jul 16 15:36:15.494500 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:15.494461    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9bzgd" podStartSLOduration=2.982214241 podStartE2EDuration="6.494448358s" totalImagesPullingTime="3.512234117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:36:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:15.493983252 +0000 UTC m=+41.682847235" watchObservedRunningTime="2026-07-16 15:36:15.494448358 +0000 UTC m=+41.683312312"
Jul 16 15:36:15.517662 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:15.517623    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wc65r" podStartSLOduration=9.539114167 podStartE2EDuration="41.517610428s" totalImagesPullingTime="31.978496261s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:15.515854652 +0000 UTC m=+41.704718606" watchObservedRunningTime="2026-07-16 15:36:15.517610428 +0000 UTC m=+41.706474361"
Jul 16 15:36:15.531988 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:15.531923    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-klvpc" podStartSLOduration=3.029952083 podStartE2EDuration="6.531907369s" totalImagesPullingTime="3.501955286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:36:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:15.531368224 +0000 UTC m=+41.720232200" watchObservedRunningTime="2026-07-16 15:36:15.531907369 +0000 UTC m=+41.720771326"
Jul 16 15:36:16.225080 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:16.225043    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:16.438352 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:16.438314    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:16.554600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:16.554512    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:16.694535 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:16.694491    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:17.301167 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:17.301127    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:17.382424 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:17.382397    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:17.382546 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:17.382436    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:17.382582 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:17.382549    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 15:36:17.382582 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:17.382561    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75d6fbc47f-rjqcs: secret "image-registry-tls" not found
Jul 16 15:36:17.382641 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:17.382607    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 15:36:17.382688 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:17.382613    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls podName:692cf320-ee61-4f05-aa65-da1598213514 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:25.382597633 +0000 UTC m=+51.571461573 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls") pod "image-registry-75d6fbc47f-rjqcs" (UID: "692cf320-ee61-4f05-aa65-da1598213514") : secret "image-registry-tls" not found
Jul 16 15:36:17.382688 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:17.382668    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls podName:83c60a71-04d9-4ee3-8431-b381d610783c nodeName:}" failed. No retries permitted until 2026-07-16 15:36:25.382654527 +0000 UTC m=+51.571518461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls") pod "dns-default-77d2v" (UID: "83c60a71-04d9-4ee3-8431-b381d610783c") : secret "dns-default-metrics-tls" not found
Jul 16 15:36:17.482780 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:17.482751    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:17.482907 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:17.482813    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:17.482907 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:17.482874    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 15:36:17.483021 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:17.482916    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 15:36:17.483021 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:17.482922    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm podName:b7607d9f-e61a-4b41-9494-4622fb17f848 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:25.482910399 +0000 UTC m=+51.671774332 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b9qf6" (UID: "b7607d9f-e61a-4b41-9494-4622fb17f848") : configmap "kube-rbac-proxy" not found
Jul 16 15:36:17.483021 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:17.483017    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls podName:b7607d9f-e61a-4b41-9494-4622fb17f848 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:25.483001392 +0000 UTC m=+51.671865339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b9qf6" (UID: "b7607d9f-e61a-4b41-9494-4622fb17f848") : secret "insights-runtime-extractor-tls" not found
Jul 16 15:36:18.085211 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:18.085163    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:18.392320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:18.392247    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:19.003483 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:19.003453    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:19.103320 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:19.103283    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:19.203234 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:19.203209    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:19.525999 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:19.525952    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:19.927366 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:19.927330    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:20.223906 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:20.223821    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:20.325107 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:20.325071    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:20.568563 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:20.568478    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:20.675088 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:20.675048    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:20.772610 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:20.772571    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:20.870211 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:20.870161    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:20.970572 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:20.970546    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:21.112755 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:21.112716    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:21.169679 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:21.169583    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:21.242016 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:21.241985    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:21.346869 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:21.346844    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:21.443020 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:21.442917    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:21.628366 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:21.628333    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:22.025104 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:22.025058    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:22.126129 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:22.126097    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:22.187179 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:22.187147    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:22.247677 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:22.247648    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:22.442759 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:22.442731    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:22.642266 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:22.642227    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:22.744198 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:22.744109    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:22.911583 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:22.911548    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:23.059698 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:23.059625    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:23.896341 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:23.896308    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:23.997176 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:23.997148    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:24.770377 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:24.770338    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:25.115447 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:25.115418    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:25.443777 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:25.443703    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:25.443777 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:25.443745    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:25.446703 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:25.446671    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/83c60a71-04d9-4ee3-8431-b381d610783c-metrics-tls\") pod \"dns-default-77d2v\" (UID: \"83c60a71-04d9-4ee3-8431-b381d610783c\") " pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:25.446853 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:25.446835    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls\") pod \"image-registry-75d6fbc47f-rjqcs\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") " pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:25.513499 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:25.513474    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:25.530257 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:25.530225    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:25.544659 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:25.544639    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:25.544754 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:25.544694    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:25.544808 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:25.544792    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 15:36:25.544869 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:25.544859    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls podName:b7607d9f-e61a-4b41-9494-4622fb17f848 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:41.54484141 +0000 UTC m=+67.733705358 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b9qf6" (UID: "b7607d9f-e61a-4b41-9494-4622fb17f848") : secret "insights-runtime-extractor-tls" not found
Jul 16 15:36:25.545265 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:25.545248    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b7607d9f-e61a-4b41-9494-4622fb17f848-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:25.661418 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:25.661391    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75d6fbc47f-rjqcs"]
Jul 16 15:36:25.665106 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:36:25.665084    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod692cf320_ee61_4f05_aa65_da1598213514.slice/crio-82fa72e9b4a66e47c66e4cba9d352346a939ef54cefbc6317452506f72248be8 WatchSource:0}: Error finding container 82fa72e9b4a66e47c66e4cba9d352346a939ef54cefbc6317452506f72248be8: Status 404 returned error can't find the container with id 82fa72e9b4a66e47c66e4cba9d352346a939ef54cefbc6317452506f72248be8
Jul 16 15:36:25.675035 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:25.675011    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-77d2v"]
Jul 16 15:36:25.677365 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:36:25.677340    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83c60a71_04d9_4ee3_8431_b381d610783c.slice/crio-8eea91794ee9e8340dd526a6558a920c6c664a6cec539b64554bcf77dd8ef0e7 WatchSource:0}: Error finding container 8eea91794ee9e8340dd526a6558a920c6c664a6cec539b64554bcf77dd8ef0e7: Status 404 returned error can't find the container with id 8eea91794ee9e8340dd526a6558a920c6c664a6cec539b64554bcf77dd8ef0e7
Jul 16 15:36:25.879412 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:25.879389    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:25.975054 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:25.974972    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:26.510804 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:26.510771    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs" event={"ID":"692cf320-ee61-4f05-aa65-da1598213514","Type":"ContainerStarted","Data":"2b7fe16fb92ae12720d70f268c2a9916c5fecf2e133fc67c9266fcdb106695f1"}
Jul 16 15:36:26.511271 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:26.510827    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs" event={"ID":"692cf320-ee61-4f05-aa65-da1598213514","Type":"ContainerStarted","Data":"82fa72e9b4a66e47c66e4cba9d352346a939ef54cefbc6317452506f72248be8"}
Jul 16 15:36:26.511271 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:26.510953    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:26.511978 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:26.511954    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-77d2v" event={"ID":"83c60a71-04d9-4ee3-8431-b381d610783c","Type":"ContainerStarted","Data":"8eea91794ee9e8340dd526a6558a920c6c664a6cec539b64554bcf77dd8ef0e7"}
Jul 16 15:36:26.518641 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:26.518612    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:26.532271 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:26.532236    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs" podStartSLOduration=28.532219887 podStartE2EDuration="28.532219887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:26.531174845 +0000 UTC m=+52.720038793" watchObservedRunningTime="2026-07-16 15:36:26.532219887 +0000 UTC m=+52.721083825"
Jul 16 15:36:27.246636 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:27.246605    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:27.304199 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:27.304172    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:27.364293 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:27.364269    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:28.520379 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:28.520340    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-77d2v" event={"ID":"83c60a71-04d9-4ee3-8431-b381d610783c","Type":"ContainerStarted","Data":"8104e30d4b8b795529e6e11cd6eb7d6c478553790a78391384ddca14616056f3"}
Jul 16 15:36:28.520770 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:28.520393    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-77d2v" event={"ID":"83c60a71-04d9-4ee3-8431-b381d610783c","Type":"ContainerStarted","Data":"e143f58b4e6905f489af49cacbece89d22030cd69c15f3ca1e06bca77b2fa4d8"}
Jul 16 15:36:28.520770 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:28.520479    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:28.541554 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:28.541510    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-77d2v" podStartSLOduration=17.518127387 podStartE2EDuration="19.541496016s" totalImagesPullingTime="2.023368629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:36:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:28.540755254 +0000 UTC m=+54.729619209" watchObservedRunningTime="2026-07-16 15:36:28.541496016 +0000 UTC m=+54.730359972"
Jul 16 15:36:29.992217 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:29.992176    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:30.492182 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:30.492144    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:31.490628 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.490596    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-csgsh"]
Jul 16 15:36:31.495380 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.495360    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-csgsh"
Jul 16 15:36:31.499057 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.499035    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 16 15:36:31.499174 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.499035    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 16 15:36:31.499174 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.499096    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7wf78\""
Jul 16 15:36:31.505009 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.504986    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"]
Jul 16 15:36:31.508049 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.508032    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"]
Jul 16 15:36:31.508187 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.508170    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.511177 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.511159    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78998f58c4-52slj"]
Jul 16 15:36:31.511268 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.511229    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 16 15:36:31.511334 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.511291    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"
Jul 16 15:36:31.511334 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.511302    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 16 15:36:31.511969 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.511951    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 16 15:36:31.512123 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.512100    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 16 15:36:31.512270 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.512247    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 16 15:36:31.512336 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.512254    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 16 15:36:31.513246 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.513228    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 16 15:36:31.514170 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.514152    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-csgsh"]
Jul 16 15:36:31.514283 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.514253    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78998f58c4-52slj"
Jul 16 15:36:31.514366 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.514350    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 16 15:36:31.516813 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.516794    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-6w9rz\""
Jul 16 15:36:31.517315 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.517299    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 16 15:36:31.522426 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.522405    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"]
Jul 16 15:36:31.529074 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.528838    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"]
Jul 16 15:36:31.529890 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.529871    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78998f58c4-52slj"]
Jul 16 15:36:31.532210 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.532190    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75d6fbc47f-rjqcs"]
Jul 16 15:36:31.587459 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.587428    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpxxc\" (UniqueName: \"kubernetes.io/projected/33847950-a945-4737-8a00-40c2ba7ccdf9-kube-api-access-qpxxc\") pod \"managed-serviceaccount-addon-agent-78998f58c4-52slj\" (UID: \"33847950-a945-4737-8a00-40c2ba7ccdf9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78998f58c4-52slj"
Jul 16 15:36:31.587459 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.587458    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-hub\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.587640 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.587475    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.587640 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.587493    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56ee72e4-c4b3-4042-bb36-d5b239671b10-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-csgsh\" (UID: \"56ee72e4-c4b3-4042-bb36-d5b239671b10\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-csgsh"
Jul 16 15:36:31.587640 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.587546    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-ca\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.587640 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.587563    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c8a51b15-c673-43ff-94ba-99f9ddea1a9d-klusterlet-config\") pod \"klusterlet-addon-workmgr-cbb9cb694-dgtrm\" (UID: \"c8a51b15-c673-43ff-94ba-99f9ddea1a9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"
Jul 16 15:36:31.587640 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.587584    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8a51b15-c673-43ff-94ba-99f9ddea1a9d-tmp\") pod \"klusterlet-addon-workmgr-cbb9cb694-dgtrm\" (UID: \"c8a51b15-c673-43ff-94ba-99f9ddea1a9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"
Jul 16 15:36:31.587810 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.587656    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfbfv\" (UniqueName: \"kubernetes.io/projected/c8a51b15-c673-43ff-94ba-99f9ddea1a9d-kube-api-access-pfbfv\") pod \"klusterlet-addon-workmgr-cbb9cb694-dgtrm\" (UID: \"c8a51b15-c673-43ff-94ba-99f9ddea1a9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"
Jul 16 15:36:31.587810 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.587687    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/56ee72e4-c4b3-4042-bb36-d5b239671b10-nginx-conf\") pod \"networking-console-plugin-6cf856758c-csgsh\" (UID: \"56ee72e4-c4b3-4042-bb36-d5b239671b10\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-csgsh"
Jul 16 15:36:31.587810 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.587702    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.587810 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.587746    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47s6g\" (UniqueName: \"kubernetes.io/projected/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-kube-api-access-47s6g\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.587810 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.587773    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/33847950-a945-4737-8a00-40c2ba7ccdf9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78998f58c4-52slj\" (UID: \"33847950-a945-4737-8a00-40c2ba7ccdf9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78998f58c4-52slj"
Jul 16 15:36:31.587810 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.587793    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.596610 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.596591    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-756bbccf75-r2xtx"]
Jul 16 15:36:31.599760 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.599745    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.660201 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.660159    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-756bbccf75-r2xtx"]
Jul 16 15:36:31.688764 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.688723    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6721b192-01a8-44b8-ac19-b98d931a02f4-trusted-ca\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.688964 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.688778    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpxxc\" (UniqueName: \"kubernetes.io/projected/33847950-a945-4737-8a00-40c2ba7ccdf9-kube-api-access-qpxxc\") pod \"managed-serviceaccount-addon-agent-78998f58c4-52slj\" (UID: \"33847950-a945-4737-8a00-40c2ba7ccdf9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78998f58c4-52slj"
Jul 16 15:36:31.688964 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.688824    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6721b192-01a8-44b8-ac19-b98d931a02f4-installation-pull-secrets\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.688964 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.688859    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-hub\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.688964 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.688883    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.688964 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.688909    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56ee72e4-c4b3-4042-bb36-d5b239671b10-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-csgsh\" (UID: \"56ee72e4-c4b3-4042-bb36-d5b239671b10\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-csgsh"
Jul 16 15:36:31.689190 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.688971    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-ca\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.689190 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689000    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c8a51b15-c673-43ff-94ba-99f9ddea1a9d-klusterlet-config\") pod \"klusterlet-addon-workmgr-cbb9cb694-dgtrm\" (UID: \"c8a51b15-c673-43ff-94ba-99f9ddea1a9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"
Jul 16 15:36:31.689190 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689025    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8a51b15-c673-43ff-94ba-99f9ddea1a9d-tmp\") pod \"klusterlet-addon-workmgr-cbb9cb694-dgtrm\" (UID: \"c8a51b15-c673-43ff-94ba-99f9ddea1a9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"
Jul 16 15:36:31.689190 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689157    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfbfv\" (UniqueName: \"kubernetes.io/projected/c8a51b15-c673-43ff-94ba-99f9ddea1a9d-kube-api-access-pfbfv\") pod \"klusterlet-addon-workmgr-cbb9cb694-dgtrm\" (UID: \"c8a51b15-c673-43ff-94ba-99f9ddea1a9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"
Jul 16 15:36:31.689389 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689193    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6721b192-01a8-44b8-ac19-b98d931a02f4-ca-trust-extracted\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.689389 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689232    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/56ee72e4-c4b3-4042-bb36-d5b239671b10-nginx-conf\") pod \"networking-console-plugin-6cf856758c-csgsh\" (UID: \"56ee72e4-c4b3-4042-bb36-d5b239671b10\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-csgsh"
Jul 16 15:36:31.689389 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689259    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.689389 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689291    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twqnm\" (UniqueName: \"kubernetes.io/projected/6721b192-01a8-44b8-ac19-b98d931a02f4-kube-api-access-twqnm\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.689389 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689318    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6721b192-01a8-44b8-ac19-b98d931a02f4-bound-sa-token\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.689389 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689373    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47s6g\" (UniqueName: \"kubernetes.io/projected/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-kube-api-access-47s6g\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.689694 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689399    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8a51b15-c673-43ff-94ba-99f9ddea1a9d-tmp\") pod \"klusterlet-addon-workmgr-cbb9cb694-dgtrm\" (UID: \"c8a51b15-c673-43ff-94ba-99f9ddea1a9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"
Jul 16 15:36:31.689694 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689401    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/33847950-a945-4737-8a00-40c2ba7ccdf9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78998f58c4-52slj\" (UID: \"33847950-a945-4737-8a00-40c2ba7ccdf9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78998f58c4-52slj"
Jul 16 15:36:31.689694 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689457    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6721b192-01a8-44b8-ac19-b98d931a02f4-registry-tls\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.689694 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689486    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.689694 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689515    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6721b192-01a8-44b8-ac19-b98d931a02f4-image-registry-private-configuration\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.689694 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.689546    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6721b192-01a8-44b8-ac19-b98d931a02f4-registry-certificates\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.690351 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.690318    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.690458 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.690433    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/56ee72e4-c4b3-4042-bb36-d5b239671b10-nginx-conf\") pod \"networking-console-plugin-6cf856758c-csgsh\" (UID: \"56ee72e4-c4b3-4042-bb36-d5b239671b10\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-csgsh"
Jul 16 15:36:31.692071 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.692050    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.692203 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.692059    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c8a51b15-c673-43ff-94ba-99f9ddea1a9d-klusterlet-config\") pod \"klusterlet-addon-workmgr-cbb9cb694-dgtrm\" (UID: \"c8a51b15-c673-43ff-94ba-99f9ddea1a9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"
Jul 16 15:36:31.692270 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.692249    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56ee72e4-c4b3-4042-bb36-d5b239671b10-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-csgsh\" (UID: \"56ee72e4-c4b3-4042-bb36-d5b239671b10\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-csgsh"
Jul 16 15:36:31.692462 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.692442    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-ca\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.692498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.692487    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/33847950-a945-4737-8a00-40c2ba7ccdf9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78998f58c4-52slj\" (UID: \"33847950-a945-4737-8a00-40c2ba7ccdf9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78998f58c4-52slj"
Jul 16 15:36:31.692619 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.692601    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-hub\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.693000 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.692980    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.720279 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.720251    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfbfv\" (UniqueName: \"kubernetes.io/projected/c8a51b15-c673-43ff-94ba-99f9ddea1a9d-kube-api-access-pfbfv\") pod \"klusterlet-addon-workmgr-cbb9cb694-dgtrm\" (UID: \"c8a51b15-c673-43ff-94ba-99f9ddea1a9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"
Jul 16 15:36:31.720444 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.720424    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpxxc\" (UniqueName: \"kubernetes.io/projected/33847950-a945-4737-8a00-40c2ba7ccdf9-kube-api-access-qpxxc\") pod \"managed-serviceaccount-addon-agent-78998f58c4-52slj\" (UID: \"33847950-a945-4737-8a00-40c2ba7ccdf9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78998f58c4-52slj"
Jul 16 15:36:31.728284 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.728262    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47s6g\" (UniqueName: \"kubernetes.io/projected/894b2837-5d5e-43d2-bd3e-1c1749ea2d0c-kube-api-access-47s6g\") pod \"cluster-proxy-proxy-agent-555c4bc9cc-dpstl\" (UID: \"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.790429 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.790354    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6721b192-01a8-44b8-ac19-b98d931a02f4-installation-pull-secrets\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.790555 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.790456    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6721b192-01a8-44b8-ac19-b98d931a02f4-ca-trust-extracted\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.790555 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.790494    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twqnm\" (UniqueName: \"kubernetes.io/projected/6721b192-01a8-44b8-ac19-b98d931a02f4-kube-api-access-twqnm\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.790555 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.790521    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6721b192-01a8-44b8-ac19-b98d931a02f4-bound-sa-token\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.790702 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.790579    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6721b192-01a8-44b8-ac19-b98d931a02f4-registry-tls\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.790702 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.790613    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6721b192-01a8-44b8-ac19-b98d931a02f4-image-registry-private-configuration\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.790814 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.790722    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6721b192-01a8-44b8-ac19-b98d931a02f4-registry-certificates\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.790814 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.790763    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6721b192-01a8-44b8-ac19-b98d931a02f4-trusted-ca\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.790949 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.790877    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6721b192-01a8-44b8-ac19-b98d931a02f4-ca-trust-extracted\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.791563 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.791535    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6721b192-01a8-44b8-ac19-b98d931a02f4-registry-certificates\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.791749 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.791732    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6721b192-01a8-44b8-ac19-b98d931a02f4-trusted-ca\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.792554 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.792535    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6721b192-01a8-44b8-ac19-b98d931a02f4-installation-pull-secrets\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.792668 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.792653    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6721b192-01a8-44b8-ac19-b98d931a02f4-image-registry-private-configuration\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.792820 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.792803    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6721b192-01a8-44b8-ac19-b98d931a02f4-registry-tls\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.803279 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.803253    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6721b192-01a8-44b8-ac19-b98d931a02f4-bound-sa-token\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.803724 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.803707    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twqnm\" (UniqueName: \"kubernetes.io/projected/6721b192-01a8-44b8-ac19-b98d931a02f4-kube-api-access-twqnm\") pod \"image-registry-756bbccf75-r2xtx\" (UID: \"6721b192-01a8-44b8-ac19-b98d931a02f4\") " pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.803785 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.803773    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-csgsh"
Jul 16 15:36:31.823984 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.823964    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"
Jul 16 15:36:31.833054 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.833032    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"
Jul 16 15:36:31.838082 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.838066    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78998f58c4-52slj"
Jul 16 15:36:31.869986 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.867385    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:31.913694 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.913385    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:31.956779 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.956746    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-csgsh"]
Jul 16 15:36:31.957282 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:31.957090    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:31.967054 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:36:31.964159    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56ee72e4_c4b3_4042_bb36_d5b239671b10.slice/crio-f65c5e11deb15dc3151d9bc33ee3a97e8cb69a62320d4a9e2e7b4343953be010 WatchSource:0}: Error finding container f65c5e11deb15dc3151d9bc33ee3a97e8cb69a62320d4a9e2e7b4343953be010: Status 404 returned error can't find the container with id f65c5e11deb15dc3151d9bc33ee3a97e8cb69a62320d4a9e2e7b4343953be010
Jul 16 15:36:32.023651 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:32.023411    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:32.023651 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:32.023550    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl"]
Jul 16 15:36:32.026200 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:36:32.026168    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod894b2837_5d5e_43d2_bd3e_1c1749ea2d0c.slice/crio-9de354f48e908bc30737c78b0a8e7f083678328d16ea446071b5821130712702 WatchSource:0}: Error finding container 9de354f48e908bc30737c78b0a8e7f083678328d16ea446071b5821130712702: Status 404 returned error can't find the container with id 9de354f48e908bc30737c78b0a8e7f083678328d16ea446071b5821130712702
Jul 16 15:36:32.040862 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:32.040837    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78998f58c4-52slj"]
Jul 16 15:36:32.043109 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:36:32.043082    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33847950_a945_4737_8a00_40c2ba7ccdf9.slice/crio-a8dd83ad0036dd01f289e309ef216b926401a80bb447eb0f9d936d4adb56a294 WatchSource:0}: Error finding container a8dd83ad0036dd01f289e309ef216b926401a80bb447eb0f9d936d4adb56a294: Status 404 returned error can't find the container with id a8dd83ad0036dd01f289e309ef216b926401a80bb447eb0f9d936d4adb56a294
Jul 16 15:36:32.043395 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:32.043375    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"]
Jul 16 15:36:32.046007 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:36:32.045982    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8a51b15_c673_43ff_94ba_99f9ddea1a9d.slice/crio-86821c81540a8e856449250759ca6bff694d30060eaa8fbc862471bfc15472a4 WatchSource:0}: Error finding container 86821c81540a8e856449250759ca6bff694d30060eaa8fbc862471bfc15472a4: Status 404 returned error can't find the container with id 86821c81540a8e856449250759ca6bff694d30060eaa8fbc862471bfc15472a4
Jul 16 15:36:32.089002 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:32.088975    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-756bbccf75-r2xtx"]
Jul 16 15:36:32.092002 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:36:32.091967    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6721b192_01a8_44b8_ac19_b98d931a02f4.slice/crio-ba1c825f886b735075ef1bd9cacdb739067a00e4d8a47b5b7b62c2c803ccb348 WatchSource:0}: Error finding container ba1c825f886b735075ef1bd9cacdb739067a00e4d8a47b5b7b62c2c803ccb348: Status 404 returned error can't find the container with id ba1c825f886b735075ef1bd9cacdb739067a00e4d8a47b5b7b62c2c803ccb348
Jul 16 15:36:32.092306 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:32.092264    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:32.536946 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:32.536885    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl" event={"ID":"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c","Type":"ContainerStarted","Data":"9de354f48e908bc30737c78b0a8e7f083678328d16ea446071b5821130712702"}
Jul 16 15:36:32.537976 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:32.537949    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-csgsh" event={"ID":"56ee72e4-c4b3-4042-bb36-d5b239671b10","Type":"ContainerStarted","Data":"f65c5e11deb15dc3151d9bc33ee3a97e8cb69a62320d4a9e2e7b4343953be010"}
Jul 16 15:36:32.539348 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:32.539326    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-756bbccf75-r2xtx" event={"ID":"6721b192-01a8-44b8-ac19-b98d931a02f4","Type":"ContainerStarted","Data":"9bea5e13f038b7353238e8d41f27370c51015440cec8554c81a47872d159d04d"}
Jul 16 15:36:32.539440 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:32.539367    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-756bbccf75-r2xtx" event={"ID":"6721b192-01a8-44b8-ac19-b98d931a02f4","Type":"ContainerStarted","Data":"ba1c825f886b735075ef1bd9cacdb739067a00e4d8a47b5b7b62c2c803ccb348"}
Jul 16 15:36:32.539440 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:32.539427    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:32.540375 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:32.540353    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78998f58c4-52slj" event={"ID":"33847950-a945-4737-8a00-40c2ba7ccdf9","Type":"ContainerStarted","Data":"a8dd83ad0036dd01f289e309ef216b926401a80bb447eb0f9d936d4adb56a294"}
Jul 16 15:36:32.541326 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:32.541308    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm" event={"ID":"c8a51b15-c673-43ff-94ba-99f9ddea1a9d","Type":"ContainerStarted","Data":"86821c81540a8e856449250759ca6bff694d30060eaa8fbc862471bfc15472a4"}
Jul 16 15:36:32.559075 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:32.559033    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-756bbccf75-r2xtx" podStartSLOduration=1.559021617 podStartE2EDuration="1.559021617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:36:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:32.558287497 +0000 UTC m=+58.747151472" watchObservedRunningTime="2026-07-16 15:36:32.559021617 +0000 UTC m=+58.747885595"
Jul 16 15:36:33.690527 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:33.690493    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:33.791976 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:33.791899    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:33.876437 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:33.876315    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:34.008077 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:34.007925    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:34.189177 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:34.189140    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:34.443281 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:34.443257    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sltmp"
Jul 16 15:36:34.551765 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:34.551726    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-csgsh" event={"ID":"56ee72e4-c4b3-4042-bb36-d5b239671b10","Type":"ContainerStarted","Data":"754553ed8d564b9ad4170880425901a8b72ee65a91b3ac9d7177df9f5f45f83f"}
Jul 16 15:36:34.568236 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:34.568183    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-csgsh" podStartSLOduration=1.759035903 podStartE2EDuration="3.568165968s" totalImagesPullingTime="1.809130065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:34.568157072 +0000 UTC m=+60.757021040" watchObservedRunningTime="2026-07-16 15:36:34.568165968 +0000 UTC m=+60.757029924"
Jul 16 15:36:34.598105 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:34.598078    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:34.658332 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:34.658300    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:34.727627 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:34.727548    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:34.828633 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:34.828602    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:34.889961 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:34.889902    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:36.511179 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:36.511140    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:36.876046 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:36.876012    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:36.976004 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:36.975978    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:37.157480 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:37.157395    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:37.274143 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:37.274115    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:37.566057 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:37.565968    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78998f58c4-52slj" event={"ID":"33847950-a945-4737-8a00-40c2ba7ccdf9","Type":"ContainerStarted","Data":"1f3075f14f2db8a61b6cb4e34429ad66dcde141c314aa7e21938ebe50f9272b0"}
Jul 16 15:36:37.567313 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:37.567285    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm" event={"ID":"c8a51b15-c673-43ff-94ba-99f9ddea1a9d","Type":"ContainerStarted","Data":"93730b79e6541d7f5b29536c671624b2db6515e70b4db5426c4d784b11459b92"}
Jul 16 15:36:37.567479 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:37.567395    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"
Jul 16 15:36:37.569079 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:37.569059    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm"
Jul 16 15:36:37.590274 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:37.590201    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78998f58c4-52slj" podStartSLOduration=2.062793829 podStartE2EDuration="6.590185662s" totalImagesPullingTime="4.527391833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:37.589437051 +0000 UTC m=+63.778301009" watchObservedRunningTime="2026-07-16 15:36:37.590185662 +0000 UTC m=+63.779049618"
Jul 16 15:36:37.607291 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:37.607254    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbb9cb694-dgtrm" podStartSLOduration=2.070077806 podStartE2EDuration="6.607243716s" totalImagesPullingTime="4.53716591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:37.606193325 +0000 UTC m=+63.795057290" watchObservedRunningTime="2026-07-16 15:36:37.607243716 +0000 UTC m=+63.796107665"
Jul 16 15:36:37.962033 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:37.961997    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:38.049056 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:38.049027    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:38.144467 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:38.144434    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:38.352267 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:38.352237    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:38.440036 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:38.440001    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:38.525676 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:38.525654    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-77d2v"
Jul 16 15:36:38.924859 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:38.924825    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:39.007277 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:39.007248    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:39.109164 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:39.109135    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:39.960375 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:39.960333    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs\") pod \"network-metrics-daemon-98ggn\" (UID: \"52710c6b-b4c4-48db-9637-2f94d8c959bb\") " pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:36:39.964920 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:39.964894    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 15:36:39.972653 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:39.972635    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52710c6b-b4c4-48db-9637-2f94d8c959bb-metrics-certs\") pod \"network-metrics-daemon-98ggn\" (UID: \"52710c6b-b4c4-48db-9637-2f94d8c959bb\") " pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:36:40.061022 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.060996    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:36:40.064131 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.064116    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 15:36:40.073490 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.073475    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/83adbeaf-0de4-4fec-be65-5a9a8b9ce326-original-pull-secret\") pod \"global-pull-secret-syncer-4t8xm\" (UID: \"83adbeaf-0de4-4fec-be65-5a9a8b9ce326\") " pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:36:40.077139 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.077123    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zfqhq\""
Jul 16 15:36:40.084813 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.084798    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98ggn"
Jul 16 15:36:40.086995 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.086979    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t8xm"
Jul 16 15:36:40.161469 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.161356    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2fnt\" (UniqueName: \"kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt\") pod \"network-check-target-8jdhr\" (UID: \"958875c8-4f2f-474f-8201-b9090b15b099\") " pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:36:40.164750 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.164726    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 15:36:40.174984 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.174958    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 15:36:40.184897 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.184868    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2fnt\" (UniqueName: \"kubernetes.io/projected/958875c8-4f2f-474f-8201-b9090b15b099-kube-api-access-n2fnt\") pod \"network-check-target-8jdhr\" (UID: \"958875c8-4f2f-474f-8201-b9090b15b099\") " pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:36:40.225243 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.224634    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4t8xm"]
Jul 16 15:36:40.240698 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.240635    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-98ggn"]
Jul 16 15:36:40.242849 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:36:40.242827    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52710c6b_b4c4_48db_9637_2f94d8c959bb.slice/crio-2af1a5209b56398f963b6a3fa0ab5ab12d3cfccef09cf3ba0c9f913ef6bda692 WatchSource:0}: Error finding container 2af1a5209b56398f963b6a3fa0ab5ab12d3cfccef09cf3ba0c9f913ef6bda692: Status 404 returned error can't find the container with id 2af1a5209b56398f963b6a3fa0ab5ab12d3cfccef09cf3ba0c9f913ef6bda692
Jul 16 15:36:40.301696 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.301668    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:40.384520 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.384500    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2scpx\""
Jul 16 15:36:40.391216 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.391199    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:36:40.515633 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.515568    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8jdhr"]
Jul 16 15:36:40.518185 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:36:40.518162    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod958875c8_4f2f_474f_8201_b9090b15b099.slice/crio-459bbba27b9db8c4e0f18fc9e394d605ba1d0c9d108de1b2baec60d6efed5dc9 WatchSource:0}: Error finding container 459bbba27b9db8c4e0f18fc9e394d605ba1d0c9d108de1b2baec60d6efed5dc9: Status 404 returned error can't find the container with id 459bbba27b9db8c4e0f18fc9e394d605ba1d0c9d108de1b2baec60d6efed5dc9
Jul 16 15:36:40.578385 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.578352    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8jdhr" event={"ID":"958875c8-4f2f-474f-8201-b9090b15b099","Type":"ContainerStarted","Data":"459bbba27b9db8c4e0f18fc9e394d605ba1d0c9d108de1b2baec60d6efed5dc9"}
Jul 16 15:36:40.579418 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.579395    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4t8xm" event={"ID":"83adbeaf-0de4-4fec-be65-5a9a8b9ce326","Type":"ContainerStarted","Data":"6d99dfa3d422a98622306aea5d367c259bde53251cb6ad288536556a9dbe2ab8"}
Jul 16 15:36:40.580468 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:40.580448    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-98ggn" event={"ID":"52710c6b-b4c4-48db-9637-2f94d8c959bb","Type":"ContainerStarted","Data":"2af1a5209b56398f963b6a3fa0ab5ab12d3cfccef09cf3ba0c9f913ef6bda692"}
Jul 16 15:36:41.539155 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:41.539125    2570 patch_prober.go:28] interesting pod/image-registry-75d6fbc47f-rjqcs container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 16 15:36:41.539449 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:41.539176    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs" podUID="692cf320-ee61-4f05-aa65-da1598213514" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 15:36:41.572138 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:41.572084    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:41.575081 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:41.575033    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b7607d9f-e61a-4b41-9494-4622fb17f848-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b9qf6\" (UID: \"b7607d9f-e61a-4b41-9494-4622fb17f848\") " pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:41.589000 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:41.588968    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-98ggn" event={"ID":"52710c6b-b4c4-48db-9637-2f94d8c959bb","Type":"ContainerStarted","Data":"bcb3b6512311e5072d8821adc38e50b4ef228757c04939d95b8f244960d1e2d5"}
Jul 16 15:36:41.772288 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:41.772209    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zzsd4\""
Jul 16 15:36:41.779633 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:41.779611    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b9qf6"
Jul 16 15:36:41.937744 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:41.937715    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b9qf6"]
Jul 16 15:36:41.940972 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:36:41.940946    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7607d9f_e61a_4b41_9494_4622fb17f848.slice/crio-56e6db48ec86e8084850e0734cb9b8cd7c7553cb7747acc93e46c95e4557f05b WatchSource:0}: Error finding container 56e6db48ec86e8084850e0734cb9b8cd7c7553cb7747acc93e46c95e4557f05b: Status 404 returned error can't find the container with id 56e6db48ec86e8084850e0734cb9b8cd7c7553cb7747acc93e46c95e4557f05b
Jul 16 15:36:42.168485 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:42.168446    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:42.268413 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:42.268379    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:42.598113 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:42.597240    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-98ggn" event={"ID":"52710c6b-b4c4-48db-9637-2f94d8c959bb","Type":"ContainerStarted","Data":"888d55d6aea6973061506c21de1e863a26037ca0c8d55e65880be5f738fc585c"}
Jul 16 15:36:42.600613 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:42.600540    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b9qf6" event={"ID":"b7607d9f-e61a-4b41-9494-4622fb17f848","Type":"ContainerStarted","Data":"56e6db48ec86e8084850e0734cb9b8cd7c7553cb7747acc93e46c95e4557f05b"}
Jul 16 15:36:42.668546 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:42.668495    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:42.770668 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:42.770633    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:42.870885 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:42.870797    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:43.611974 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:43.611815    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:43.880025 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:43.879926    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:43.935604 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:43.935504    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:44.538074 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:44.538033    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:44.639466 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:44.639431    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:45.074402 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.074362    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:45.252201 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.252155    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:45.328052 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.327913    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:45.335751 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.335693    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-98ggn" podStartSLOduration=70.133958358 podStartE2EDuration="1m11.335671667s" totalImagesPullingTime="1.201713309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:42.616922978 +0000 UTC m=+68.805786933" watchObservedRunningTime="2026-07-16 15:36:45.335671667 +0000 UTC m=+71.524535623"
Jul 16 15:36:45.337689 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.336463    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bkkkh"]
Jul 16 15:36:45.355843 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.355820    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.358908 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.358736    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 16 15:36:45.358908 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.358745    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4qtrk\""
Jul 16 15:36:45.358908 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.358750    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 16 15:36:45.358908 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.358844    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 16 15:36:45.359698 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.359669    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 16 15:36:45.359698 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.359683    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 16 15:36:45.359875 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.359811    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 16 15:36:45.400078 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.400051    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.400197 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.400090    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-tls\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.400197 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.400115    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/09f00530-b743-432e-a12f-af9301dbd46a-root\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.400276 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.400190    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-accelerators-collector-config\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.400276 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.400242    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jgjb\" (UniqueName: \"kubernetes.io/projected/09f00530-b743-432e-a12f-af9301dbd46a-kube-api-access-6jgjb\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.400345 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.400296    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09f00530-b743-432e-a12f-af9301dbd46a-sys\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.400345 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.400328    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-wtmp\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.400416 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.400388    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-textfile\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.400449 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.400418    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09f00530-b743-432e-a12f-af9301dbd46a-metrics-client-ca\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.414411 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.414378    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:45.501600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.501569    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09f00530-b743-432e-a12f-af9301dbd46a-sys\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.501617    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-wtmp\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.501663    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-textfile\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.501672    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09f00530-b743-432e-a12f-af9301dbd46a-sys\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.501689    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09f00530-b743-432e-a12f-af9301dbd46a-metrics-client-ca\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.501724    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.501813    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-wtmp\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.501861    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-tls\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.501893    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/09f00530-b743-432e-a12f-af9301dbd46a-root\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.501926    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-accelerators-collector-config\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.502004    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jgjb\" (UniqueName: \"kubernetes.io/projected/09f00530-b743-432e-a12f-af9301dbd46a-kube-api-access-6jgjb\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.502067    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-textfile\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.502120    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/09f00530-b743-432e-a12f-af9301dbd46a-root\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:45.502190    2570 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:36:45.502240    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-tls podName:09f00530-b743-432e-a12f-af9301dbd46a nodeName:}" failed. No retries permitted until 2026-07-16 15:36:46.002223454 +0000 UTC m=+72.191087397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-tls") pod "node-exporter-bkkkh" (UID: "09f00530-b743-432e-a12f-af9301dbd46a") : secret "node-exporter-tls" not found
Jul 16 15:36:45.502506 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.502458    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09f00530-b743-432e-a12f-af9301dbd46a-metrics-client-ca\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.503095 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.502608    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-accelerators-collector-config\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.504708 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.504687    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.514370 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.514325    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jgjb\" (UniqueName: \"kubernetes.io/projected/09f00530-b743-432e-a12f-af9301dbd46a-kube-api-access-6jgjb\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:45.515966 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.515920    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:45.616031 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.616000    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:45.914763 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:45.914654    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:46.007025 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:46.006986    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-tls\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:46.009490 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:46.009465    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/09f00530-b743-432e-a12f-af9301dbd46a-node-exporter-tls\") pod \"node-exporter-bkkkh\" (UID: \"09f00530-b743-432e-a12f-af9301dbd46a\") " pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:46.017169 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:46.017140    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:46.116495 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:46.116458    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:46.268162 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:46.268079    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bkkkh"
Jul 16 15:36:46.478830 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:46.478796    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:46.538560 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:46.538411    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:46.927291 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:46.927256    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:47.021170 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:47.021128    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:47.641451 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:47.641418    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:47.741701 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:47.741648    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:48.395691 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:48.395653    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:48.450399 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:48.450362    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:48.958249 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:48.958219    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:49.058689 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:49.058585    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:49.323036 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:36:49.323009    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09f00530_b743_432e_a12f_af9301dbd46a.slice/crio-450bcd8ae7fc815f75e322ac7113ce8e7c091b9ff56a669aad949f90e6a8bd56 WatchSource:0}: Error finding container 450bcd8ae7fc815f75e322ac7113ce8e7c091b9ff56a669aad949f90e6a8bd56: Status 404 returned error can't find the container with id 450bcd8ae7fc815f75e322ac7113ce8e7c091b9ff56a669aad949f90e6a8bd56
Jul 16 15:36:49.630290 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:49.630243    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bkkkh" event={"ID":"09f00530-b743-432e-a12f-af9301dbd46a","Type":"ContainerStarted","Data":"450bcd8ae7fc815f75e322ac7113ce8e7c091b9ff56a669aad949f90e6a8bd56"}
Jul 16 15:36:49.632069 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:49.632004    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8jdhr" event={"ID":"958875c8-4f2f-474f-8201-b9090b15b099","Type":"ContainerStarted","Data":"4e262ac2ec5f378b789db5b565ce10a5008a5b41cd48395a97a73caa290fa075"}
Jul 16 15:36:49.632296 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:49.632276    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:36:49.633828 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:49.633803    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4t8xm" event={"ID":"83adbeaf-0de4-4fec-be65-5a9a8b9ce326","Type":"ContainerStarted","Data":"9e14d32f82703c4bb9b041527ddfebe801bd21e93ed4c02a29204dbae0e0c7b6"}
Jul 16 15:36:49.635611 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:49.635590    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b9qf6" event={"ID":"b7607d9f-e61a-4b41-9494-4622fb17f848","Type":"ContainerStarted","Data":"ab525cf9d9de5053940a27c79c78ae052929413755a7c0e8e3746af0e61131c8"}
Jul 16 15:36:49.652082 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:49.652046    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8jdhr" podStartSLOduration=66.852569664 podStartE2EDuration="1m15.652033179s" totalImagesPullingTime="8.799463515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:49.650705768 +0000 UTC m=+75.839569752" watchObservedRunningTime="2026-07-16 15:36:49.652033179 +0000 UTC m=+75.840897135"
Jul 16 15:36:49.677479 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:49.677423    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4t8xm" podStartSLOduration=66.584826025 podStartE2EDuration="1m15.677410631s" totalImagesPullingTime="9.092584606s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:49.676337507 +0000 UTC m=+75.865201461" watchObservedRunningTime="2026-07-16 15:36:49.677410631 +0000 UTC m=+75.866274588"
Jul 16 15:36:50.117550 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:50.117516    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:50.207852 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:50.207820    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:50.440895 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:50.440813    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:51.539761 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:51.539736    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:51.652204 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:51.651307    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl" event={"ID":"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c","Type":"ContainerStarted","Data":"48d3b5430be4a1e6271072072890b2985cb18a088776fe8a5d954fb2b5339cab"}
Jul 16 15:36:51.652204 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:51.651358    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl" event={"ID":"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c","Type":"ContainerStarted","Data":"75ebf56edc07eae33e3dddd2c5f2c1c584b2e1889d09a0954ec12ef40807db09"}
Jul 16 15:36:51.652204 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:51.651379    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl" event={"ID":"894b2837-5d5e-43d2-bd3e-1c1749ea2d0c","Type":"ContainerStarted","Data":"a97f8d97766cf605249fd4521be481a748eb859f04695e80df21f253f8aa17e7"}
Jul 16 15:36:51.734944 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:51.734870    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-555c4bc9cc-dpstl" podStartSLOduration=1.392732907 podStartE2EDuration="20.734848765s" totalImagesPullingTime="19.342115858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:51.733436276 +0000 UTC m=+77.922300232" watchObservedRunningTime="2026-07-16 15:36:51.734848765 +0000 UTC m=+77.923712720"
Jul 16 15:36:52.267943 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:52.267894    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:52.656790 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:52.656753    2570 generic.go:356] "Generic (PLEG): container finished" podID="09f00530-b743-432e-a12f-af9301dbd46a" containerID="ceda50d005ac405446e2799900c1e6fffe46789466f9d58aa874879cdf6793a1" exitCode=0
Jul 16 15:36:52.657255 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:52.656854    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bkkkh" event={"ID":"09f00530-b743-432e-a12f-af9301dbd46a","Type":"ContainerDied","Data":"ceda50d005ac405446e2799900c1e6fffe46789466f9d58aa874879cdf6793a1"}
Jul 16 15:36:53.550910 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:53.550881    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-756bbccf75-r2xtx"
Jul 16 15:36:53.701746 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:53.701710    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:54.107829 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:54.107795    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:54.667246 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:54.667209    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bkkkh" event={"ID":"09f00530-b743-432e-a12f-af9301dbd46a","Type":"ContainerStarted","Data":"fa9ddc6a6b743a7cda9e99f3a758d9a4a06c6c0a18a43cec9027e67aef612a10"}
Jul 16 15:36:54.667412 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:54.667317    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bkkkh" event={"ID":"09f00530-b743-432e-a12f-af9301dbd46a","Type":"ContainerStarted","Data":"edb85d0cd80fcb2f0dc88328b1c0cceb54b4f21e5adb39742dc7e81765816c65"}
Jul 16 15:36:54.668849 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:54.668826    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b9qf6" event={"ID":"b7607d9f-e61a-4b41-9494-4622fb17f848","Type":"ContainerStarted","Data":"559dce40564b3a81d41bbdcd2f62481cec2a798138b9d5f920a3a0f180421a4f"}
Jul 16 15:36:54.691229 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:54.691151    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bkkkh" podStartSLOduration=6.902336747 podStartE2EDuration="9.691135411s" totalImagesPullingTime="2.715237964s" totalInitContainerRuntime="73.5607ms" isStatefulPod=true podCreationTimestamp="2026-07-16 15:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:54.689263825 +0000 UTC m=+80.878127780" watchObservedRunningTime="2026-07-16 15:36:54.691135411 +0000 UTC m=+80.879999365"
Jul 16 15:36:55.962261 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:55.962129    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:56.551901 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.551861    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs" podUID="692cf320-ee61-4f05-aa65-da1598213514" containerName="registry" containerID="cri-o://2b7fe16fb92ae12720d70f268c2a9916c5fecf2e133fc67c9266fcdb106695f1" gracePeriod=30
Jul 16 15:36:56.682156 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.682117    2570 generic.go:356] "Generic (PLEG): container finished" podID="692cf320-ee61-4f05-aa65-da1598213514" containerID="2b7fe16fb92ae12720d70f268c2a9916c5fecf2e133fc67c9266fcdb106695f1" exitCode=0
Jul 16 15:36:56.682319 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.682200    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs" event={"ID":"692cf320-ee61-4f05-aa65-da1598213514","Type":"ContainerDied","Data":"2b7fe16fb92ae12720d70f268c2a9916c5fecf2e133fc67c9266fcdb106695f1"}
Jul 16 15:36:56.761897 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.761839    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:56.805964 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.805883    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:56.836923 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.836892    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:36:56.892849 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.892823    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-bound-sa-token\") pod \"692cf320-ee61-4f05-aa65-da1598213514\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") "
Jul 16 15:36:56.893017 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.892862    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhzmn\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-kube-api-access-lhzmn\") pod \"692cf320-ee61-4f05-aa65-da1598213514\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") "
Jul 16 15:36:56.893017 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.892906    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/692cf320-ee61-4f05-aa65-da1598213514-ca-trust-extracted\") pod \"692cf320-ee61-4f05-aa65-da1598213514\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") "
Jul 16 15:36:56.893017 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.892966    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/692cf320-ee61-4f05-aa65-da1598213514-image-registry-private-configuration\") pod \"692cf320-ee61-4f05-aa65-da1598213514\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") "
Jul 16 15:36:56.893194 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.893160    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls\") pod \"692cf320-ee61-4f05-aa65-da1598213514\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") "
Jul 16 15:36:56.893248 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.893209    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/692cf320-ee61-4f05-aa65-da1598213514-registry-certificates\") pod \"692cf320-ee61-4f05-aa65-da1598213514\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") "
Jul 16 15:36:56.893303 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.893267    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/692cf320-ee61-4f05-aa65-da1598213514-installation-pull-secrets\") pod \"692cf320-ee61-4f05-aa65-da1598213514\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") "
Jul 16 15:36:56.893350 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.893316    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/692cf320-ee61-4f05-aa65-da1598213514-trusted-ca\") pod \"692cf320-ee61-4f05-aa65-da1598213514\" (UID: \"692cf320-ee61-4f05-aa65-da1598213514\") "
Jul 16 15:36:56.894066 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.893983    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/692cf320-ee61-4f05-aa65-da1598213514-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "692cf320-ee61-4f05-aa65-da1598213514" (UID: "692cf320-ee61-4f05-aa65-da1598213514"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 15:36:56.894199 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.894090    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/692cf320-ee61-4f05-aa65-da1598213514-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "692cf320-ee61-4f05-aa65-da1598213514" (UID: "692cf320-ee61-4f05-aa65-da1598213514"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 15:36:56.896729 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.896656    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "692cf320-ee61-4f05-aa65-da1598213514" (UID: "692cf320-ee61-4f05-aa65-da1598213514"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:36:56.896729 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.896680    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/692cf320-ee61-4f05-aa65-da1598213514-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "692cf320-ee61-4f05-aa65-da1598213514" (UID: "692cf320-ee61-4f05-aa65-da1598213514"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 15:36:56.896982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.896920    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/692cf320-ee61-4f05-aa65-da1598213514-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "692cf320-ee61-4f05-aa65-da1598213514" (UID: "692cf320-ee61-4f05-aa65-da1598213514"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 15:36:56.897094 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.897051    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "692cf320-ee61-4f05-aa65-da1598213514" (UID: "692cf320-ee61-4f05-aa65-da1598213514"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:36:56.897356 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.897325    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-kube-api-access-lhzmn" (OuterVolumeSpecName: "kube-api-access-lhzmn") pod "692cf320-ee61-4f05-aa65-da1598213514" (UID: "692cf320-ee61-4f05-aa65-da1598213514"). InnerVolumeSpecName "kube-api-access-lhzmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:36:56.904315 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.904292    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/692cf320-ee61-4f05-aa65-da1598213514-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "692cf320-ee61-4f05-aa65-da1598213514" (UID: "692cf320-ee61-4f05-aa65-da1598213514"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 15:36:56.993846 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.993817    2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-bound-sa-token\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:36:56.993846 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.993846    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhzmn\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-kube-api-access-lhzmn\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:36:56.994289 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.993861    2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/692cf320-ee61-4f05-aa65-da1598213514-ca-trust-extracted\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:36:56.994289 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.993870    2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/692cf320-ee61-4f05-aa65-da1598213514-image-registry-private-configuration\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:36:56.994289 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.993879    2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/692cf320-ee61-4f05-aa65-da1598213514-registry-tls\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:36:56.994289 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.993887    2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/692cf320-ee61-4f05-aa65-da1598213514-registry-certificates\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:36:56.994289 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.993895    2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/692cf320-ee61-4f05-aa65-da1598213514-installation-pull-secrets\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:36:56.994289 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:56.993902    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/692cf320-ee61-4f05-aa65-da1598213514-trusted-ca\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:36:57.687990 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:57.687957    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs" event={"ID":"692cf320-ee61-4f05-aa65-da1598213514","Type":"ContainerDied","Data":"82fa72e9b4a66e47c66e4cba9d352346a939ef54cefbc6317452506f72248be8"}
Jul 16 15:36:57.687990 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:57.687990    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75d6fbc47f-rjqcs"
Jul 16 15:36:57.688221 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:57.688028    2570 scope.go:117] "RemoveContainer" containerID="2b7fe16fb92ae12720d70f268c2a9916c5fecf2e133fc67c9266fcdb106695f1"
Jul 16 15:36:57.711897 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:57.711872    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75d6fbc47f-rjqcs"]
Jul 16 15:36:57.716273 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:57.716249    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-75d6fbc47f-rjqcs"]
Jul 16 15:36:58.266214 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:58.266175    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="692cf320-ee61-4f05-aa65-da1598213514" path="/var/lib/kubelet/pods/692cf320-ee61-4f05-aa65-da1598213514/volumes"
Jul 16 15:36:59.697407 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:59.697370    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b9qf6" event={"ID":"b7607d9f-e61a-4b41-9494-4622fb17f848","Type":"ContainerStarted","Data":"2e22858f3ba01d37a57de5ad355c8dc85820ca62ef98480c13acf198ae5f7fa7"}
Jul 16 15:36:59.716701 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:59.716655    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-b9qf6" podStartSLOduration=33.514901653 podStartE2EDuration="50.716639186s" totalImagesPullingTime="17.201737533s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:36:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:59.715154999 +0000 UTC m=+85.904018955" watchObservedRunningTime="2026-07-16 15:36:59.716639186 +0000 UTC m=+85.905503188"
Jul 16 15:36:59.986041 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:36:59.985953    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:00.084173 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:00.084146    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:01.755301 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:01.755266    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:01.837492 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:01.837445    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:01.985110 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:01.985063    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:02.083554 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:02.083474    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:02.184236 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:02.184183    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:02.309515 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:02.309464    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:02.865629 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:02.865572    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:03.403415 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:03.403384    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:03.955325 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:03.955285    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:04.056209 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:04.056171    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:04.156150 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:04.156117    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:04.691993 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:04.691958    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:04.745079 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:04.745047    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:05.851819 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:05.851782    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:05.912408 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:05.912379    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:11.751667 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:11.751635    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:11.852399 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:11.852351    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:15.105542 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:15.105500    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:15.205441 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:15.205406    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:15.257286 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:15.257259    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:15.679873 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:15.679814    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:15.734550 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:15.734512    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:15.797081 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:15.797053    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:20.642348 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:20.642312    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8jdhr"
Jul 16 15:37:27.701632 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:27.701597    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:27.788392 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:27.788351    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:29.996612 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:29.996563    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:30.107235 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:30.107194    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:36.270364 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:36.270262    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:37:43.746888 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:37:43.746856    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:38:01.653207 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:38:01.653171    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:38:01.708581 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:38:01.708544    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:38:08.066518 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:38:08.066485    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:38:08.160817 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:38:08.160783    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:38:13.615527 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:38:13.615494    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:38:13.683742 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:38:13.683709    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:38:18.964359 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:38:18.964251    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:38:19.158278 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:38:19.158240    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:38:30.017089 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:38:30.016997    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:38:30.112045 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:38:30.112010    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:38:30.212443 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:38:30.212413    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:38:55.977813 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:38:55.977780    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:38:56.065666 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:38:56.065639    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:02.258390 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:02.258351    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:02.313522 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:02.313482    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:06.037947 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:06.037899    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:22.591349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:22.591312    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:22.690285 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:22.690253    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:26.179398 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:26.179342    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:26.279298 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:26.279259    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:27.445417 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:27.445383    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:27.550464 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:27.550425    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:27.826771 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:27.826681    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:27.909076 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:27.909039    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:30.013873 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:30.013843    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:30.443486 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:30.443454    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:32.879134 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:32.879101    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:32.979129 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:32.979101    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:38.615540 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:38.615490    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:38.685128 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:38.685094    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:40.122422 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:40.122383    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:40.222727 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:40.222694    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:45.291606 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:45.291570    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:45.396950 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:45.396903    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:46.436296 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:46.436258    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:46.536088 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:46.536050    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:47.547526 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:47.547492    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:39:59.977227 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:39:59.977137    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:00.085508 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:00.085467    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:00.610952 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:00.610894    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:00.853624 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:00.853590    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:00.979831 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:00.979740    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:01.025325 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.025288    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"]
Jul 16 15:40:01.025528 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.025517    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="692cf320-ee61-4f05-aa65-da1598213514" containerName="registry"
Jul 16 15:40:01.025571 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.025529    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="692cf320-ee61-4f05-aa65-da1598213514" containerName="registry"
Jul 16 15:40:01.025604 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.025576    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="692cf320-ee61-4f05-aa65-da1598213514" containerName="registry"
Jul 16 15:40:01.028333 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.028314    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.044085 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.044061    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 16 15:40:01.047741 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.047725    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pnzkc\""
Jul 16 15:40:01.047851 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.047829    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 16 15:40:01.047971 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.047953    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 16 15:40:01.052359 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.052344    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 16 15:40:01.064279 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.064255    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"]
Jul 16 15:40:01.146448 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.146414    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh7gz\" (UniqueName: \"kubernetes.io/projected/0a7674a4-071f-4241-9adf-e3fd7567f5de-kube-api-access-qh7gz\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.146612 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.146459    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a7674a4-071f-4241-9adf-e3fd7567f5de-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.146612 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.146490    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0a7674a4-071f-4241-9adf-e3fd7567f5de-manifests\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.146612 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.146561    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a7674a4-071f-4241-9adf-e3fd7567f5de-tmp\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.146612 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.146590    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0a7674a4-071f-4241-9adf-e3fd7567f5de-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.240290 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.240201    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:01.247193 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.247158    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a7674a4-071f-4241-9adf-e3fd7567f5de-tmp\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.247264 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.247200    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0a7674a4-071f-4241-9adf-e3fd7567f5de-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.247264 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.247240    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh7gz\" (UniqueName: \"kubernetes.io/projected/0a7674a4-071f-4241-9adf-e3fd7567f5de-kube-api-access-qh7gz\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.247369 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.247265    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a7674a4-071f-4241-9adf-e3fd7567f5de-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.247369 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.247287    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0a7674a4-071f-4241-9adf-e3fd7567f5de-manifests\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.247676 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.247641    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0a7674a4-071f-4241-9adf-e3fd7567f5de-manifests\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.249307 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.249288    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a7674a4-071f-4241-9adf-e3fd7567f5de-tmp\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.249522 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.249503    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0a7674a4-071f-4241-9adf-e3fd7567f5de-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.249626 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.249609    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a7674a4-071f-4241-9adf-e3fd7567f5de-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.256727 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.256702    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh7gz\" (UniqueName: \"kubernetes.io/projected/0a7674a4-071f-4241-9adf-e3fd7567f5de-kube-api-access-qh7gz\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-qjzng\" (UID: \"0a7674a4-071f-4241-9adf-e3fd7567f5de\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.338092 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.338055    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:01.340215 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.340190    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:01.432220 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.432190    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:01.474147 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.474006    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"]
Jul 16 15:40:01.476886 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:40:01.476860    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a7674a4_071f_4241_9adf_e3fd7567f5de.slice/crio-3427eed21c8a39027b43175773dbc5ca129f160f9a65f49e831b3ff56ffbee5b WatchSource:0}: Error finding container 3427eed21c8a39027b43175773dbc5ca129f160f9a65f49e831b3ff56ffbee5b: Status 404 returned error can't find the container with id 3427eed21c8a39027b43175773dbc5ca129f160f9a65f49e831b3ff56ffbee5b
Jul 16 15:40:01.491508 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.491439    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:01.574292 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:01.574251    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:02.292481 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:02.292448    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng" event={"ID":"0a7674a4-071f-4241-9adf-e3fd7567f5de","Type":"ContainerStarted","Data":"3427eed21c8a39027b43175773dbc5ca129f160f9a65f49e831b3ff56ffbee5b"}
Jul 16 15:40:03.177926 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:03.177891    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:03.279094 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:03.279036    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:03.383630 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:03.383575    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:06.310350 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:06.310311    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng" event={"ID":"0a7674a4-071f-4241-9adf-e3fd7567f5de","Type":"ContainerStarted","Data":"4dbac8db667cc03fa803941033ab51f256eb198c75ef2d175b51456145adceb5"}
Jul 16 15:40:07.315082 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:07.315039    2570 generic.go:356] "Generic (PLEG): container finished" podID="0a7674a4-071f-4241-9adf-e3fd7567f5de" containerID="4dbac8db667cc03fa803941033ab51f256eb198c75ef2d175b51456145adceb5" exitCode=0
Jul 16 15:40:07.315469 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:07.315126    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng" event={"ID":"0a7674a4-071f-4241-9adf-e3fd7567f5de","Type":"ContainerDied","Data":"4dbac8db667cc03fa803941033ab51f256eb198c75ef2d175b51456145adceb5"}
Jul 16 15:40:09.322566 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:09.322534    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng" event={"ID":"0a7674a4-071f-4241-9adf-e3fd7567f5de","Type":"ContainerStarted","Data":"3f4f31e123c6d10a5f8481779fd8e9e026d396ab6ce33caf5a3f60e4bee0df9a"}
Jul 16 15:40:09.322956 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:09.322665    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:09.346984 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:09.346922    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng" podStartSLOduration=3.592140948 podStartE2EDuration="9.346905666s" totalImagesPullingTime="4.740177877s" totalInitContainerRuntime="1.014586841s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:40:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:40:09.345749691 +0000 UTC m=+275.534613646" watchObservedRunningTime="2026-07-16 15:40:09.346905666 +0000 UTC m=+275.535769621"
Jul 16 15:40:11.139360 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:11.139330    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:11.248669 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:11.248632    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:12.268601 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:12.268563    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:12.348668 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:12.348632    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:20.327491 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:20.327457    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-qjzng"
Jul 16 15:40:22.177951 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:22.177898    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:22.283516 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:22.283482    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:23.030074 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.030039    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:23.169568 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.169534    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh"]
Jul 16 15:40:23.184113 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.184003    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh"
Jul 16 15:40:23.185175 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.185153    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh"]
Jul 16 15:40:23.187484 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.187453    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 16 15:40:23.187598 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.187456    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 16 15:40:23.187598 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.187591    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 16 15:40:23.188515 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.188498    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 16 15:40:23.188604 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.188542    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gxtlg\""
Jul 16 15:40:23.239032 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.239004    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2p5gp"]
Jul 16 15:40:23.242130 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.242111    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2p5gp"
Jul 16 15:40:23.244631 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.244615    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 16 15:40:23.244727 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.244651    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-hzv9m\""
Jul 16 15:40:23.253228 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.253210    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2p5gp"]
Jul 16 15:40:23.312683 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.312612    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b853572-837c-44c8-bd2b-9a9919a6ca98-tls-certs\") pod \"kube-auth-proxy-5bb6d5565c-lk4jh\" (UID: \"7b853572-837c-44c8-bd2b-9a9919a6ca98\") " pod="openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh"
Jul 16 15:40:23.312822 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.312727    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fdhg\" (UniqueName: \"kubernetes.io/projected/7b853572-837c-44c8-bd2b-9a9919a6ca98-kube-api-access-2fdhg\") pod \"kube-auth-proxy-5bb6d5565c-lk4jh\" (UID: \"7b853572-837c-44c8-bd2b-9a9919a6ca98\") " pod="openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh"
Jul 16 15:40:23.312893 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.312817    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b853572-837c-44c8-bd2b-9a9919a6ca98-tmp\") pod \"kube-auth-proxy-5bb6d5565c-lk4jh\" (UID: \"7b853572-837c-44c8-bd2b-9a9919a6ca98\") " pod="openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh"
Jul 16 15:40:23.316226 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.316198    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:23.414121 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.414089    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b853572-837c-44c8-bd2b-9a9919a6ca98-tls-certs\") pod \"kube-auth-proxy-5bb6d5565c-lk4jh\" (UID: \"7b853572-837c-44c8-bd2b-9a9919a6ca98\") " pod="openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh"
Jul 16 15:40:23.414339 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.414276    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fdhg\" (UniqueName: \"kubernetes.io/projected/7b853572-837c-44c8-bd2b-9a9919a6ca98-kube-api-access-2fdhg\") pod \"kube-auth-proxy-5bb6d5565c-lk4jh\" (UID: \"7b853572-837c-44c8-bd2b-9a9919a6ca98\") " pod="openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh"
Jul 16 15:40:23.414413 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.414350    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a-cert\") pod \"odh-model-controller-7998c67dc4-2p5gp\" (UID: \"3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a\") " pod="opendatahub/odh-model-controller-7998c67dc4-2p5gp"
Jul 16 15:40:23.414413 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.414379    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b853572-837c-44c8-bd2b-9a9919a6ca98-tmp\") pod \"kube-auth-proxy-5bb6d5565c-lk4jh\" (UID: \"7b853572-837c-44c8-bd2b-9a9919a6ca98\") " pod="openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh"
Jul 16 15:40:23.414518 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.414419    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8279d\" (UniqueName: \"kubernetes.io/projected/3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a-kube-api-access-8279d\") pod \"odh-model-controller-7998c67dc4-2p5gp\" (UID: \"3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a\") " pod="opendatahub/odh-model-controller-7998c67dc4-2p5gp"
Jul 16 15:40:23.415576 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.415552    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:23.417034 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.417009    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b853572-837c-44c8-bd2b-9a9919a6ca98-tmp\") pod \"kube-auth-proxy-5bb6d5565c-lk4jh\" (UID: \"7b853572-837c-44c8-bd2b-9a9919a6ca98\") " pod="openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh"
Jul 16 15:40:23.417151 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.417133    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b853572-837c-44c8-bd2b-9a9919a6ca98-tls-certs\") pod \"kube-auth-proxy-5bb6d5565c-lk4jh\" (UID: \"7b853572-837c-44c8-bd2b-9a9919a6ca98\") " pod="openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh"
Jul 16 15:40:23.425671 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.425651    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fdhg\" (UniqueName: \"kubernetes.io/projected/7b853572-837c-44c8-bd2b-9a9919a6ca98-kube-api-access-2fdhg\") pod \"kube-auth-proxy-5bb6d5565c-lk4jh\" (UID: \"7b853572-837c-44c8-bd2b-9a9919a6ca98\") " pod="openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh"
Jul 16 15:40:23.495698 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.495669    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh"
Jul 16 15:40:23.515050 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.515026    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a-cert\") pod \"odh-model-controller-7998c67dc4-2p5gp\" (UID: \"3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a\") " pod="opendatahub/odh-model-controller-7998c67dc4-2p5gp"
Jul 16 15:40:23.515170 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.515070    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8279d\" (UniqueName: \"kubernetes.io/projected/3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a-kube-api-access-8279d\") pod \"odh-model-controller-7998c67dc4-2p5gp\" (UID: \"3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a\") " pod="opendatahub/odh-model-controller-7998c67dc4-2p5gp"
Jul 16 15:40:23.517323 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.517297    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a-cert\") pod \"odh-model-controller-7998c67dc4-2p5gp\" (UID: \"3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a\") " pod="opendatahub/odh-model-controller-7998c67dc4-2p5gp"
Jul 16 15:40:23.524428 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.524405    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8279d\" (UniqueName: \"kubernetes.io/projected/3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a-kube-api-access-8279d\") pod \"odh-model-controller-7998c67dc4-2p5gp\" (UID: \"3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a\") " pod="opendatahub/odh-model-controller-7998c67dc4-2p5gp"
Jul 16 15:40:23.551804 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.551773    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2p5gp"
Jul 16 15:40:23.619242 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.619216    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:23.627903 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.627874    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh"]
Jul 16 15:40:23.630734 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:40:23.630703    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b853572_837c_44c8_bd2b_9a9919a6ca98.slice/crio-9363d7e43aafbaab8b926aa6d2829a75b95d94d444367df67b9e07ed77ff1969 WatchSource:0}: Error finding container 9363d7e43aafbaab8b926aa6d2829a75b95d94d444367df67b9e07ed77ff1969: Status 404 returned error can't find the container with id 9363d7e43aafbaab8b926aa6d2829a75b95d94d444367df67b9e07ed77ff1969
Jul 16 15:40:23.698850 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:23.698825    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2p5gp"]
Jul 16 15:40:23.700949 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:40:23.700912    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ecda9dd_6d1a_49d3_819b_873d7b7b0e5a.slice/crio-72f899d83f83b97bd1a206f00f2e28ff83c5e53be3f4d2982dd670bd645b4852 WatchSource:0}: Error finding container 72f899d83f83b97bd1a206f00f2e28ff83c5e53be3f4d2982dd670bd645b4852: Status 404 returned error can't find the container with id 72f899d83f83b97bd1a206f00f2e28ff83c5e53be3f4d2982dd670bd645b4852
Jul 16 15:40:24.200568 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.200537    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kzl5p"]
Jul 16 15:40:24.204890 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.204872    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-kzl5p"
Jul 16 15:40:24.207472 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.207449    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 16 15:40:24.207566 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.207492    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-8kqvb\""
Jul 16 15:40:24.215701 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.215677    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kzl5p"]
Jul 16 15:40:24.320725 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.320687    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89h7x\" (UniqueName: \"kubernetes.io/projected/fafcdc9c-b525-4453-b491-839385024054-kube-api-access-89h7x\") pod \"model-serving-api-54d89bd5fc-kzl5p\" (UID: \"fafcdc9c-b525-4453-b491-839385024054\") " pod="opendatahub/model-serving-api-54d89bd5fc-kzl5p"
Jul 16 15:40:24.320891 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.320746    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fafcdc9c-b525-4453-b491-839385024054-tls-certs\") pod \"model-serving-api-54d89bd5fc-kzl5p\" (UID: \"fafcdc9c-b525-4453-b491-839385024054\") " pod="opendatahub/model-serving-api-54d89bd5fc-kzl5p"
Jul 16 15:40:24.372890 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.372856    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2p5gp" event={"ID":"3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a","Type":"ContainerStarted","Data":"72f899d83f83b97bd1a206f00f2e28ff83c5e53be3f4d2982dd670bd645b4852"}
Jul 16 15:40:24.373832 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.373809    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh" event={"ID":"7b853572-837c-44c8-bd2b-9a9919a6ca98","Type":"ContainerStarted","Data":"9363d7e43aafbaab8b926aa6d2829a75b95d94d444367df67b9e07ed77ff1969"}
Jul 16 15:40:24.404120 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.404091    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"]
Jul 16 15:40:24.407241 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.407226    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.409517 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.409480    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 16 15:40:24.409623 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.409594    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-tl9p9\""
Jul 16 15:40:24.411021 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.411005    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 16 15:40:24.411217 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.411203    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 16 15:40:24.411295 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.411206    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 16 15:40:24.419504 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.419486    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"]
Jul 16 15:40:24.421534 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.421502    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89h7x\" (UniqueName: \"kubernetes.io/projected/fafcdc9c-b525-4453-b491-839385024054-kube-api-access-89h7x\") pod \"model-serving-api-54d89bd5fc-kzl5p\" (UID: \"fafcdc9c-b525-4453-b491-839385024054\") " pod="opendatahub/model-serving-api-54d89bd5fc-kzl5p"
Jul 16 15:40:24.421631 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.421560    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fafcdc9c-b525-4453-b491-839385024054-tls-certs\") pod \"model-serving-api-54d89bd5fc-kzl5p\" (UID: \"fafcdc9c-b525-4453-b491-839385024054\") " pod="opendatahub/model-serving-api-54d89bd5fc-kzl5p"
Jul 16 15:40:24.423973 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.423924    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fafcdc9c-b525-4453-b491-839385024054-tls-certs\") pod \"model-serving-api-54d89bd5fc-kzl5p\" (UID: \"fafcdc9c-b525-4453-b491-839385024054\") " pod="opendatahub/model-serving-api-54d89bd5fc-kzl5p"
Jul 16 15:40:24.430487 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.430465    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89h7x\" (UniqueName: \"kubernetes.io/projected/fafcdc9c-b525-4453-b491-839385024054-kube-api-access-89h7x\") pod \"model-serving-api-54d89bd5fc-kzl5p\" (UID: \"fafcdc9c-b525-4453-b491-839385024054\") " pod="opendatahub/model-serving-api-54d89bd5fc-kzl5p"
Jul 16 15:40:24.521451 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.521370    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-kzl5p"
Jul 16 15:40:24.522606 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.522585    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/25015dcd-4435-4464-88e2-dddc79736207-local-certs\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.522675 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.522636    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.522711 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.522688    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-cacerts\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.522745 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.522713    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/25015dcd-4435-4464-88e2-dddc79736207-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.522745 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.522737    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccs56\" (UniqueName: \"kubernetes.io/projected/25015dcd-4435-4464-88e2-dddc79736207-kube-api-access-ccs56\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.522810 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.522766    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25015dcd-4435-4464-88e2-dddc79736207-istio-token\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.522810 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.522783    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.623732 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.623697    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.623909 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.623750    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-cacerts\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.623909 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.623767    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/25015dcd-4435-4464-88e2-dddc79736207-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.623909 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.623786    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccs56\" (UniqueName: \"kubernetes.io/projected/25015dcd-4435-4464-88e2-dddc79736207-kube-api-access-ccs56\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.623909 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.623808    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25015dcd-4435-4464-88e2-dddc79736207-istio-token\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.623909 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.623825    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.624220 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.624194    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/25015dcd-4435-4464-88e2-dddc79736207-local-certs\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.624830 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.624805    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/25015dcd-4435-4464-88e2-dddc79736207-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.626558 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.626533    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/25015dcd-4435-4464-88e2-dddc79736207-local-certs\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.626643 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.626570    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.626721 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.626701    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.626721 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.626715    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-cacerts\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.636419 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.636398    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25015dcd-4435-4464-88e2-dddc79736207-istio-token\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.645761 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.645742    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccs56\" (UniqueName: \"kubernetes.io/projected/25015dcd-4435-4464-88e2-dddc79736207-kube-api-access-ccs56\") pod \"istiod-openshift-gateway-7574459656-pn8k4\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.672274 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.672250    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kzl5p"]
Jul 16 15:40:24.674765 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:40:24.674736    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfafcdc9c_b525_4453_b491_839385024054.slice/crio-b03faa90adb3c4afece94df2327c25c47860a441a0547e3cfe7ee7e117faf057 WatchSource:0}: Error finding container b03faa90adb3c4afece94df2327c25c47860a441a0547e3cfe7ee7e117faf057: Status 404 returned error can't find the container with id b03faa90adb3c4afece94df2327c25c47860a441a0547e3cfe7ee7e117faf057
Jul 16 15:40:24.716221 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.716195    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:24.865722 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:24.865696    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"]
Jul 16 15:40:24.867950 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:40:24.867900    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25015dcd_4435_4464_88e2_dddc79736207.slice/crio-4e509ea0a8d2946d2b12804a7a2de1ba13d9e3ec4f897340ef5cc7dbf8e95d89 WatchSource:0}: Error finding container 4e509ea0a8d2946d2b12804a7a2de1ba13d9e3ec4f897340ef5cc7dbf8e95d89: Status 404 returned error can't find the container with id 4e509ea0a8d2946d2b12804a7a2de1ba13d9e3ec4f897340ef5cc7dbf8e95d89
Jul 16 15:40:25.379257 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:25.379225    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-kzl5p" event={"ID":"fafcdc9c-b525-4453-b491-839385024054","Type":"ContainerStarted","Data":"b03faa90adb3c4afece94df2327c25c47860a441a0547e3cfe7ee7e117faf057"}
Jul 16 15:40:25.380615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:25.380584    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4" event={"ID":"25015dcd-4435-4464-88e2-dddc79736207","Type":"ContainerStarted","Data":"4e509ea0a8d2946d2b12804a7a2de1ba13d9e3ec4f897340ef5cc7dbf8e95d89"}
Jul 16 15:40:25.636530 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:25.636139    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:25.740027 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:25.739970    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:25.837224 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:25.837185    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:28.395506 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:28.395432    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh" event={"ID":"7b853572-837c-44c8-bd2b-9a9919a6ca98","Type":"ContainerStarted","Data":"cfb93a2d3f2845f113b868dd5336b73ad3ad0722dadb841a6bd777275ae5eecc"}
Jul 16 15:40:28.397459 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:28.397424    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2p5gp" event={"ID":"3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a","Type":"ContainerStarted","Data":"9ab6fdec4baf6d93ae10848b5c4fe8afff2fa4fcd30687c23a5641bc0690893b"}
Jul 16 15:40:28.397593 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:28.397483    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2p5gp"
Jul 16 15:40:28.399139 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:28.399073    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-kzl5p" event={"ID":"fafcdc9c-b525-4453-b491-839385024054","Type":"ContainerStarted","Data":"fe39ebfe8c26f72c04b63f66f9e829c508d5589d63887c3ae8e26b50e79330c7"}
Jul 16 15:40:28.399139 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:28.399133    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-kzl5p"
Jul 16 15:40:28.416706 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:28.416650    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5bb6d5565c-lk4jh" podStartSLOduration=0.832573737 podStartE2EDuration="5.416630428s" totalImagesPullingTime="4.584056691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:40:28.414952787 +0000 UTC m=+294.603816756" watchObservedRunningTime="2026-07-16 15:40:28.416630428 +0000 UTC m=+294.605494385"
Jul 16 15:40:28.457436 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:28.456706    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2p5gp" podStartSLOduration=0.897845509 podStartE2EDuration="5.456683316s" totalImagesPullingTime="4.558837807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:40:28.434910288 +0000 UTC m=+294.623774244" watchObservedRunningTime="2026-07-16 15:40:28.456683316 +0000 UTC m=+294.645547266"
Jul 16 15:40:28.458794 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:28.458628    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-kzl5p" podStartSLOduration=0.869540543 podStartE2EDuration="4.458609739s" totalImagesPullingTime="3.589069196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:40:28.455479383 +0000 UTC m=+294.644343363" watchObservedRunningTime="2026-07-16 15:40:28.458609739 +0000 UTC m=+294.647473695"
Jul 16 15:40:29.817998 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:29.817924    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 15:40:29.818265 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:29.818034    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 15:40:30.407826 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:30.407790    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4" event={"ID":"25015dcd-4435-4464-88e2-dddc79736207","Type":"ContainerStarted","Data":"5080dc07b76e8ed8cfaa379a313a6d38fff3d6352cd56f35e71b88365192fa54"}
Jul 16 15:40:30.408063 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:30.407986    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:30.409657 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:30.409636    2570 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-pn8k4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 15:40:30.409765 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:30.409678    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4" podUID="25015dcd-4435-4464-88e2-dddc79736207" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 15:40:30.444012 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:30.443965    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4" podStartSLOduration=1.496552386 podStartE2EDuration="6.443949141s" totalImagesPullingTime="4.947396755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:40:30.442604786 +0000 UTC m=+296.631468754" watchObservedRunningTime="2026-07-16 15:40:30.443949141 +0000 UTC m=+296.632813090"
Jul 16 15:40:30.776505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:30.776392    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:31.411680 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.411650    2570 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-pn8k4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 15:40:31.412093 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.411702    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4" podUID="25015dcd-4435-4464-88e2-dddc79736207" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 15:40:31.821793 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.821706    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"]
Jul 16 15:40:31.825022 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.824995    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:31.831541 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.831300    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-sb6db\""
Jul 16 15:40:31.848449 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.848421    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"]
Jul 16 15:40:31.989341 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.989303    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnl4k\" (UniqueName: \"kubernetes.io/projected/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-kube-api-access-vnl4k\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:31.989341 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.989338    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:31.989549 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.989364    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:31.989549 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.989423    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:31.989549 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.989493    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:31.989549 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.989527    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:31.989549 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.989550    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:31.989724 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.989586    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:31.989724 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:31.989639    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.090516 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.090485    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.090688 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.090528    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.090688 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.090566    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.090688 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.090594    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.090688 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.090632    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnl4k\" (UniqueName: \"kubernetes.io/projected/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-kube-api-access-vnl4k\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.090688 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.090658    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.090979 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.090690    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.090979 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.090719    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.090979 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.090769    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.091136 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.091001    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.091136 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.091013    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.091304 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.091260    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.091430 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.091345    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.091490 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.091447    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.093661 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.093636    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.093831 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.093813    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.109862 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.109829    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.123208 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.123172    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnl4k\" (UniqueName: \"kubernetes.io/projected/e3b4d2a6-fa24-4c3a-898a-7e35613dac00-kube-api-access-vnl4k\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47m88r\" (UID: \"e3b4d2a6-fa24-4c3a-898a-7e35613dac00\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.135177 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.135147    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:32.349845 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.349818    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"]
Jul 16 15:40:32.352089 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:40:32.352062    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3b4d2a6_fa24_4c3a_898a_7e35613dac00.slice/crio-5377d8b8d0504c6bef1a0e31afb443e8f7b32a7e733141004e4b98603a140c07 WatchSource:0}: Error finding container 5377d8b8d0504c6bef1a0e31afb443e8f7b32a7e733141004e4b98603a140c07: Status 404 returned error can't find the container with id 5377d8b8d0504c6bef1a0e31afb443e8f7b32a7e733141004e4b98603a140c07
Jul 16 15:40:32.417780 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:32.417746    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r" event={"ID":"e3b4d2a6-fa24-4c3a-898a-7e35613dac00","Type":"ContainerStarted","Data":"5377d8b8d0504c6bef1a0e31afb443e8f7b32a7e733141004e4b98603a140c07"}
Jul 16 15:40:33.598395 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:33.598355    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:34.144820 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.144783    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 15:40:34.145013 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.144849    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 15:40:34.152249 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.152221    2570 kubelet.go:1648] "Image garbage collection succeeded"
Jul 16 15:40:34.412461 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.412368    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:34.529285 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.529259    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"]
Jul 16 15:40:34.529454 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.529431    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4" podUID="25015dcd-4435-4464-88e2-dddc79736207" containerName="discovery" containerID="cri-o://5080dc07b76e8ed8cfaa379a313a6d38fff3d6352cd56f35e71b88365192fa54" gracePeriod=30
Jul 16 15:40:34.583711 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.583672    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:34.691666 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.691597    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qj26h"]
Jul 16 15:40:34.696590 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.696531    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:34.701988 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.701774    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qj26h"
Jul 16 15:40:34.703066 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.703044    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qj26h"]
Jul 16 15:40:34.705599 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.705580    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-sqzc8\""
Jul 16 15:40:34.706790 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.706629    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 16 15:40:34.784398 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.784303    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:34.811493 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.811466    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqtmp\" (UniqueName: \"kubernetes.io/projected/df6431f6-77da-4f5d-8330-ec9a1a08d6ab-kube-api-access-xqtmp\") pod \"kserve-controller-manager-d6f4b65b8-qj26h\" (UID: \"df6431f6-77da-4f5d-8330-ec9a1a08d6ab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qj26h"
Jul 16 15:40:34.811735 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.811658    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/df6431f6-77da-4f5d-8330-ec9a1a08d6ab-cert\") pod \"kserve-controller-manager-d6f4b65b8-qj26h\" (UID: \"df6431f6-77da-4f5d-8330-ec9a1a08d6ab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qj26h"
Jul 16 15:40:34.827011 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.826990    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:34.912702 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.912671    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/df6431f6-77da-4f5d-8330-ec9a1a08d6ab-cert\") pod \"kserve-controller-manager-d6f4b65b8-qj26h\" (UID: \"df6431f6-77da-4f5d-8330-ec9a1a08d6ab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qj26h"
Jul 16 15:40:34.912898 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.912732    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqtmp\" (UniqueName: \"kubernetes.io/projected/df6431f6-77da-4f5d-8330-ec9a1a08d6ab-kube-api-access-xqtmp\") pod \"kserve-controller-manager-d6f4b65b8-qj26h\" (UID: \"df6431f6-77da-4f5d-8330-ec9a1a08d6ab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qj26h"
Jul 16 15:40:34.913002 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:40:34.912914    2570 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 16 15:40:34.913068 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:40:34.913043    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6431f6-77da-4f5d-8330-ec9a1a08d6ab-cert podName:df6431f6-77da-4f5d-8330-ec9a1a08d6ab nodeName:}" failed. No retries permitted until 2026-07-16 15:40:35.413013604 +0000 UTC m=+301.601877544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/df6431f6-77da-4f5d-8330-ec9a1a08d6ab-cert") pod "kserve-controller-manager-d6f4b65b8-qj26h" (UID: "df6431f6-77da-4f5d-8330-ec9a1a08d6ab") : secret "kserve-webhook-server-cert" not found
Jul 16 15:40:34.934210 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.934170    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:34.935567 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:34.935539    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqtmp\" (UniqueName: \"kubernetes.io/projected/df6431f6-77da-4f5d-8330-ec9a1a08d6ab-kube-api-access-xqtmp\") pod \"kserve-controller-manager-d6f4b65b8-qj26h\" (UID: \"df6431f6-77da-4f5d-8330-ec9a1a08d6ab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qj26h"
Jul 16 15:40:35.013905 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.013836    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-istio-csr-dns-cert\") pod \"25015dcd-4435-4464-88e2-dddc79736207\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") "
Jul 16 15:40:35.013905 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.013893    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-cacerts\") pod \"25015dcd-4435-4464-88e2-dddc79736207\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") "
Jul 16 15:40:35.014122 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.013914    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25015dcd-4435-4464-88e2-dddc79736207-istio-token\") pod \"25015dcd-4435-4464-88e2-dddc79736207\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") "
Jul 16 15:40:35.014122 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.013964    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccs56\" (UniqueName: \"kubernetes.io/projected/25015dcd-4435-4464-88e2-dddc79736207-kube-api-access-ccs56\") pod \"25015dcd-4435-4464-88e2-dddc79736207\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") "
Jul 16 15:40:35.014122 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.013995    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/25015dcd-4435-4464-88e2-dddc79736207-istio-csr-ca-configmap\") pod \"25015dcd-4435-4464-88e2-dddc79736207\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") "
Jul 16 15:40:35.014122 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.014048    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-istio-kubeconfig\") pod \"25015dcd-4435-4464-88e2-dddc79736207\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") "
Jul 16 15:40:35.014122 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.014102    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/25015dcd-4435-4464-88e2-dddc79736207-local-certs\") pod \"25015dcd-4435-4464-88e2-dddc79736207\" (UID: \"25015dcd-4435-4464-88e2-dddc79736207\") "
Jul 16 15:40:35.014605 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.014576    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25015dcd-4435-4464-88e2-dddc79736207-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "25015dcd-4435-4464-88e2-dddc79736207" (UID: "25015dcd-4435-4464-88e2-dddc79736207"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 15:40:35.017290 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.017026    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25015dcd-4435-4464-88e2-dddc79736207-local-certs" (OuterVolumeSpecName: "local-certs") pod "25015dcd-4435-4464-88e2-dddc79736207" (UID: "25015dcd-4435-4464-88e2-dddc79736207"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 15:40:35.017290 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.017039    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25015dcd-4435-4464-88e2-dddc79736207-kube-api-access-ccs56" (OuterVolumeSpecName: "kube-api-access-ccs56") pod "25015dcd-4435-4464-88e2-dddc79736207" (UID: "25015dcd-4435-4464-88e2-dddc79736207"). InnerVolumeSpecName "kube-api-access-ccs56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:40:35.017447 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.017359    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25015dcd-4435-4464-88e2-dddc79736207-istio-token" (OuterVolumeSpecName: "istio-token") pod "25015dcd-4435-4464-88e2-dddc79736207" (UID: "25015dcd-4435-4464-88e2-dddc79736207"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:40:35.017447 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.017382    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-cacerts" (OuterVolumeSpecName: "cacerts") pod "25015dcd-4435-4464-88e2-dddc79736207" (UID: "25015dcd-4435-4464-88e2-dddc79736207"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 15:40:35.017624 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.017587    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "25015dcd-4435-4464-88e2-dddc79736207" (UID: "25015dcd-4435-4464-88e2-dddc79736207"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 15:40:35.017805 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.017771    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "25015dcd-4435-4464-88e2-dddc79736207" (UID: "25015dcd-4435-4464-88e2-dddc79736207"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 15:40:35.023126 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.023102    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:35.115255 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.115220    2570 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-cacerts\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:40:35.115255 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.115254    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25015dcd-4435-4464-88e2-dddc79736207-istio-token\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:40:35.115499 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.115270    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccs56\" (UniqueName: \"kubernetes.io/projected/25015dcd-4435-4464-88e2-dddc79736207-kube-api-access-ccs56\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:40:35.115499 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.115284    2570 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/25015dcd-4435-4464-88e2-dddc79736207-istio-csr-ca-configmap\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:40:35.115499 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.115297    2570 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-istio-kubeconfig\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:40:35.115499 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.115309    2570 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/25015dcd-4435-4464-88e2-dddc79736207-local-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:40:35.115499 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.115319    2570 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/25015dcd-4435-4464-88e2-dddc79736207-istio-csr-dns-cert\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:40:35.262184 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.262144    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:35.352420 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.352394    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:35.418063 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.418032    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/df6431f6-77da-4f5d-8330-ec9a1a08d6ab-cert\") pod \"kserve-controller-manager-d6f4b65b8-qj26h\" (UID: \"df6431f6-77da-4f5d-8330-ec9a1a08d6ab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qj26h"
Jul 16 15:40:35.420745 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.420713    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/df6431f6-77da-4f5d-8330-ec9a1a08d6ab-cert\") pod \"kserve-controller-manager-d6f4b65b8-qj26h\" (UID: \"df6431f6-77da-4f5d-8330-ec9a1a08d6ab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qj26h"
Jul 16 15:40:35.432776 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.432743    2570 generic.go:356] "Generic (PLEG): container finished" podID="25015dcd-4435-4464-88e2-dddc79736207" containerID="5080dc07b76e8ed8cfaa379a313a6d38fff3d6352cd56f35e71b88365192fa54" exitCode=0
Jul 16 15:40:35.432912 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.432799    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"
Jul 16 15:40:35.432912 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.432810    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4" event={"ID":"25015dcd-4435-4464-88e2-dddc79736207","Type":"ContainerDied","Data":"5080dc07b76e8ed8cfaa379a313a6d38fff3d6352cd56f35e71b88365192fa54"}
Jul 16 15:40:35.432912 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.432885    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4" event={"ID":"25015dcd-4435-4464-88e2-dddc79736207","Type":"ContainerDied","Data":"4e509ea0a8d2946d2b12804a7a2de1ba13d9e3ec4f897340ef5cc7dbf8e95d89"}
Jul 16 15:40:35.433136 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.432914    2570 scope.go:117] "RemoveContainer" containerID="5080dc07b76e8ed8cfaa379a313a6d38fff3d6352cd56f35e71b88365192fa54"
Jul 16 15:40:35.458920 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.458895    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"]
Jul 16 15:40:35.465880 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.465858    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pn8k4"]
Jul 16 15:40:35.617347 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.617282    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qj26h"
Jul 16 15:40:35.849944 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.849896    2570 scope.go:117] "RemoveContainer" containerID="5080dc07b76e8ed8cfaa379a313a6d38fff3d6352cd56f35e71b88365192fa54"
Jul 16 15:40:35.850311 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:40:35.850250    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5080dc07b76e8ed8cfaa379a313a6d38fff3d6352cd56f35e71b88365192fa54\": container with ID starting with 5080dc07b76e8ed8cfaa379a313a6d38fff3d6352cd56f35e71b88365192fa54 not found: ID does not exist" containerID="5080dc07b76e8ed8cfaa379a313a6d38fff3d6352cd56f35e71b88365192fa54"
Jul 16 15:40:35.850311 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.850275    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5080dc07b76e8ed8cfaa379a313a6d38fff3d6352cd56f35e71b88365192fa54"} err="failed to get container status \"5080dc07b76e8ed8cfaa379a313a6d38fff3d6352cd56f35e71b88365192fa54\": rpc error: code = NotFound desc = could not find container \"5080dc07b76e8ed8cfaa379a313a6d38fff3d6352cd56f35e71b88365192fa54\": container with ID starting with 5080dc07b76e8ed8cfaa379a313a6d38fff3d6352cd56f35e71b88365192fa54 not found: ID does not exist"
Jul 16 15:40:35.930725 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.930682    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 15:40:35.930827 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.930769    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 15:40:35.930827 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.930801    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 15:40:35.992707 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.992681    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qj26h"]
Jul 16 15:40:35.995436 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:40:35.995412    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf6431f6_77da_4f5d_8330_ec9a1a08d6ab.slice/crio-dabfe1929661732413f938032e319393ba8651e592401c0c8ba5ee05cc520c58 WatchSource:0}: Error finding container dabfe1929661732413f938032e319393ba8651e592401c0c8ba5ee05cc520c58: Status 404 returned error can't find the container with id dabfe1929661732413f938032e319393ba8651e592401c0c8ba5ee05cc520c58
Jul 16 15:40:35.996667 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:35.996649    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 15:40:36.266692 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:36.266600    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25015dcd-4435-4464-88e2-dddc79736207" path="/var/lib/kubelet/pods/25015dcd-4435-4464-88e2-dddc79736207/volumes"
Jul 16 15:40:36.439039 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:36.439001    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r" event={"ID":"e3b4d2a6-fa24-4c3a-898a-7e35613dac00","Type":"ContainerStarted","Data":"41675318e9ce2fc28f0c901928c71ba1df77d12643b71c8f33d642ba3ed10228"}
Jul 16 15:40:36.440105 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:36.440083    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qj26h" event={"ID":"df6431f6-77da-4f5d-8330-ec9a1a08d6ab","Type":"ContainerStarted","Data":"dabfe1929661732413f938032e319393ba8651e592401c0c8ba5ee05cc520c58"}
Jul 16 15:40:36.481946 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:36.481887    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r" podStartSLOduration=1.90541749 podStartE2EDuration="5.48187065s" totalImagesPullingTime="3.57645316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:40:36.48036588 +0000 UTC m=+302.669229834" watchObservedRunningTime="2026-07-16 15:40:36.48187065 +0000 UTC m=+302.670734605"
Jul 16 15:40:37.136282 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:37.136244    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:37.141894 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:37.141868    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:37.444905 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:37.444831    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:37.445984 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:37.445958    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47m88r"
Jul 16 15:40:39.405473 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:39.405444    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2p5gp"
Jul 16 15:40:39.407338 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:39.407318    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-kzl5p"
Jul 16 15:40:39.455384 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:39.455349    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qj26h" event={"ID":"df6431f6-77da-4f5d-8330-ec9a1a08d6ab","Type":"ContainerStarted","Data":"e3047eb992f615b41c656f35ede5027676829ebeaf3c08230f2766b438552176"}
Jul 16 15:40:39.455644 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:39.455624    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qj26h"
Jul 16 15:40:39.480541 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:39.480495    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qj26h" podStartSLOduration=2.577187943 podStartE2EDuration="5.480480778s" totalImagesPullingTime="2.903292835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:40:39.478514617 +0000 UTC m=+305.667378571" watchObservedRunningTime="2026-07-16 15:40:39.480480778 +0000 UTC m=+305.669344733"
Jul 16 15:40:44.095175 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:44.095131    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:44.338942 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:44.338892    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:45.850689 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:45.850654    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:40:45.911175 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:40:45.911143    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:41:11.468424 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:11.468394    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qj26h"
Jul 16 15:41:23.660164 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:23.660125    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:41:23.747427 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:23.747395    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:41:23.783089 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:23.783050    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zgqrc"]
Jul 16 15:41:23.783452 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:23.783434    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25015dcd-4435-4464-88e2-dddc79736207" containerName="discovery"
Jul 16 15:41:23.783546 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:23.783454    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="25015dcd-4435-4464-88e2-dddc79736207" containerName="discovery"
Jul 16 15:41:23.783546 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:23.783534    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="25015dcd-4435-4464-88e2-dddc79736207" containerName="discovery"
Jul 16 15:41:23.786500 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:23.786480    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zgqrc"
Jul 16 15:41:23.789208 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:23.789183    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 15:41:23.790049 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:23.790028    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 15:41:23.790161 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:23.790050    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-b888t\""
Jul 16 15:41:23.801561 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:23.801532    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zgqrc"]
Jul 16 15:41:23.903125 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:23.903095    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9vr6\" (UniqueName: \"kubernetes.io/projected/86fc9439-8986-4076-a3ca-230751d25f46-kube-api-access-r9vr6\") pod \"kuadrant-operator-catalog-zgqrc\" (UID: \"86fc9439-8986-4076-a3ca-230751d25f46\") " pod="kuadrant-system/kuadrant-operator-catalog-zgqrc"
Jul 16 15:41:24.004483 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.004379    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9vr6\" (UniqueName: \"kubernetes.io/projected/86fc9439-8986-4076-a3ca-230751d25f46-kube-api-access-r9vr6\") pod \"kuadrant-operator-catalog-zgqrc\" (UID: \"86fc9439-8986-4076-a3ca-230751d25f46\") " pod="kuadrant-system/kuadrant-operator-catalog-zgqrc"
Jul 16 15:41:24.015230 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.015201    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9vr6\" (UniqueName: \"kubernetes.io/projected/86fc9439-8986-4076-a3ca-230751d25f46-kube-api-access-r9vr6\") pod \"kuadrant-operator-catalog-zgqrc\" (UID: \"86fc9439-8986-4076-a3ca-230751d25f46\") " pod="kuadrant-system/kuadrant-operator-catalog-zgqrc"
Jul 16 15:41:24.097114 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.097076    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zgqrc"
Jul 16 15:41:24.148837 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.148798    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zgqrc"]
Jul 16 15:41:24.229637 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.229484    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zgqrc"]
Jul 16 15:41:24.231921 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:41:24.231887    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86fc9439_8986_4076_a3ca_230751d25f46.slice/crio-99eb8d0037ea98c84bb1e15399c7c504f49234e6bbcc00406c5d35dff4f99398 WatchSource:0}: Error finding container 99eb8d0037ea98c84bb1e15399c7c504f49234e6bbcc00406c5d35dff4f99398: Status 404 returned error can't find the container with id 99eb8d0037ea98c84bb1e15399c7c504f49234e6bbcc00406c5d35dff4f99398
Jul 16 15:41:24.357505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.357474    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-85qw6"]
Jul 16 15:41:24.361739 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.361723    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-85qw6"
Jul 16 15:41:24.369003 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.368977    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-85qw6"]
Jul 16 15:41:24.507684 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.507653    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrg97\" (UniqueName: \"kubernetes.io/projected/a600d138-c9a7-4b42-b67b-7ebb67fcb59a-kube-api-access-mrg97\") pod \"kuadrant-operator-catalog-85qw6\" (UID: \"a600d138-c9a7-4b42-b67b-7ebb67fcb59a\") " pod="kuadrant-system/kuadrant-operator-catalog-85qw6"
Jul 16 15:41:24.508268 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.508235    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:41:24.608790 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.608704    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrg97\" (UniqueName: \"kubernetes.io/projected/a600d138-c9a7-4b42-b67b-7ebb67fcb59a-kube-api-access-mrg97\") pod \"kuadrant-operator-catalog-85qw6\" (UID: \"a600d138-c9a7-4b42-b67b-7ebb67fcb59a\") " pod="kuadrant-system/kuadrant-operator-catalog-85qw6"
Jul 16 15:41:24.618042 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.618012    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrg97\" (UniqueName: \"kubernetes.io/projected/a600d138-c9a7-4b42-b67b-7ebb67fcb59a-kube-api-access-mrg97\") pod \"kuadrant-operator-catalog-85qw6\" (UID: \"a600d138-c9a7-4b42-b67b-7ebb67fcb59a\") " pod="kuadrant-system/kuadrant-operator-catalog-85qw6"
Jul 16 15:41:24.628488 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.628464    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zgqrc" event={"ID":"86fc9439-8986-4076-a3ca-230751d25f46","Type":"ContainerStarted","Data":"99eb8d0037ea98c84bb1e15399c7c504f49234e6bbcc00406c5d35dff4f99398"}
Jul 16 15:41:24.672228 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.672203    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-85qw6"
Jul 16 15:41:24.806533 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:24.806510    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-85qw6"]
Jul 16 15:41:24.808788 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:41:24.808756    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda600d138_c9a7_4b42_b67b_7ebb67fcb59a.slice/crio-95e1c8edadc0dbe07ced9f46cc2e301acf9cfde25548966ba4aab5a22c58cf08 WatchSource:0}: Error finding container 95e1c8edadc0dbe07ced9f46cc2e301acf9cfde25548966ba4aab5a22c58cf08: Status 404 returned error can't find the container with id 95e1c8edadc0dbe07ced9f46cc2e301acf9cfde25548966ba4aab5a22c58cf08
Jul 16 15:41:25.635393 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:25.635356    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-85qw6" event={"ID":"a600d138-c9a7-4b42-b67b-7ebb67fcb59a","Type":"ContainerStarted","Data":"95e1c8edadc0dbe07ced9f46cc2e301acf9cfde25548966ba4aab5a22c58cf08"}
Jul 16 15:41:27.644613 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:27.644572    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zgqrc" event={"ID":"86fc9439-8986-4076-a3ca-230751d25f46","Type":"ContainerStarted","Data":"6f10d2299751455fc01b1a6fbd60eae5dd6e711fdb93be47d0f1d755805950fd"}
Jul 16 15:41:27.644613 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:27.644604    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-zgqrc" podUID="86fc9439-8986-4076-a3ca-230751d25f46" containerName="registry-server" containerID="cri-o://6f10d2299751455fc01b1a6fbd60eae5dd6e711fdb93be47d0f1d755805950fd" gracePeriod=2
Jul 16 15:41:27.646079 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:27.646057    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-85qw6" event={"ID":"a600d138-c9a7-4b42-b67b-7ebb67fcb59a","Type":"ContainerStarted","Data":"372c32ba6a8ada543ab15e2e8097419977aa585448bf8d68fdc697e473acc994"}
Jul 16 15:41:27.679045 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:27.679007    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-zgqrc" podStartSLOduration=2.322606036 podStartE2EDuration="4.678991935s" totalImagesPullingTime="2.356385899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:41:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:41:27.677714813 +0000 UTC m=+353.866578780" watchObservedRunningTime="2026-07-16 15:41:27.678991935 +0000 UTC m=+353.867855891"
Jul 16 15:41:27.715032 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:27.714992    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-85qw6" podStartSLOduration=1.93251861 podStartE2EDuration="3.714979965s" totalImagesPullingTime="1.782461355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:41:27.71107655 +0000 UTC m=+353.899940505" watchObservedRunningTime="2026-07-16 15:41:27.714979965 +0000 UTC m=+353.903843920"
Jul 16 15:41:27.887985 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:27.887961    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zgqrc"
Jul 16 15:41:28.040288 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:28.040203    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9vr6\" (UniqueName: \"kubernetes.io/projected/86fc9439-8986-4076-a3ca-230751d25f46-kube-api-access-r9vr6\") pod \"86fc9439-8986-4076-a3ca-230751d25f46\" (UID: \"86fc9439-8986-4076-a3ca-230751d25f46\") "
Jul 16 15:41:28.042354 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:28.042332    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86fc9439-8986-4076-a3ca-230751d25f46-kube-api-access-r9vr6" (OuterVolumeSpecName: "kube-api-access-r9vr6") pod "86fc9439-8986-4076-a3ca-230751d25f46" (UID: "86fc9439-8986-4076-a3ca-230751d25f46"). InnerVolumeSpecName "kube-api-access-r9vr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:41:28.141104 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:28.141077    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9vr6\" (UniqueName: \"kubernetes.io/projected/86fc9439-8986-4076-a3ca-230751d25f46-kube-api-access-r9vr6\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:41:28.651533 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:28.651500    2570 generic.go:356] "Generic (PLEG): container finished" podID="86fc9439-8986-4076-a3ca-230751d25f46" containerID="6f10d2299751455fc01b1a6fbd60eae5dd6e711fdb93be47d0f1d755805950fd" exitCode=0
Jul 16 15:41:28.651998 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:28.651540    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zgqrc"
Jul 16 15:41:28.651998 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:28.651585    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zgqrc" event={"ID":"86fc9439-8986-4076-a3ca-230751d25f46","Type":"ContainerDied","Data":"6f10d2299751455fc01b1a6fbd60eae5dd6e711fdb93be47d0f1d755805950fd"}
Jul 16 15:41:28.651998 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:28.651636    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zgqrc" event={"ID":"86fc9439-8986-4076-a3ca-230751d25f46","Type":"ContainerDied","Data":"99eb8d0037ea98c84bb1e15399c7c504f49234e6bbcc00406c5d35dff4f99398"}
Jul 16 15:41:28.651998 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:28.651657    2570 scope.go:117] "RemoveContainer" containerID="6f10d2299751455fc01b1a6fbd60eae5dd6e711fdb93be47d0f1d755805950fd"
Jul 16 15:41:28.659727 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:28.659711    2570 scope.go:117] "RemoveContainer" containerID="6f10d2299751455fc01b1a6fbd60eae5dd6e711fdb93be47d0f1d755805950fd"
Jul 16 15:41:28.659974 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:41:28.659958    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f10d2299751455fc01b1a6fbd60eae5dd6e711fdb93be47d0f1d755805950fd\": container with ID starting with 6f10d2299751455fc01b1a6fbd60eae5dd6e711fdb93be47d0f1d755805950fd not found: ID does not exist" containerID="6f10d2299751455fc01b1a6fbd60eae5dd6e711fdb93be47d0f1d755805950fd"
Jul 16 15:41:28.660034 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:28.659982    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f10d2299751455fc01b1a6fbd60eae5dd6e711fdb93be47d0f1d755805950fd"} err="failed to get container status \"6f10d2299751455fc01b1a6fbd60eae5dd6e711fdb93be47d0f1d755805950fd\": rpc error: code = NotFound desc = could not find container \"6f10d2299751455fc01b1a6fbd60eae5dd6e711fdb93be47d0f1d755805950fd\": container with ID starting with 6f10d2299751455fc01b1a6fbd60eae5dd6e711fdb93be47d0f1d755805950fd not found: ID does not exist"
Jul 16 15:41:28.669612 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:28.669589    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zgqrc"]
Jul 16 15:41:28.676071 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:28.676053    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zgqrc"]
Jul 16 15:41:30.266136 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:30.266103    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86fc9439-8986-4076-a3ca-230751d25f46" path="/var/lib/kubelet/pods/86fc9439-8986-4076-a3ca-230751d25f46/volumes"
Jul 16 15:41:34.672276 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:34.672248    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-85qw6"
Jul 16 15:41:34.672708 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:34.672306    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-85qw6"
Jul 16 15:41:34.695732 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:34.695704    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-85qw6"
Jul 16 15:41:35.701234 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:35.701207    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-85qw6"
Jul 16 15:41:52.326103 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:52.326063    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:41:52.808211 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:52.808174    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:41:56.246862 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:56.246832    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9lgck"]
Jul 16 15:41:56.247327 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:56.247134    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86fc9439-8986-4076-a3ca-230751d25f46" containerName="registry-server"
Jul 16 15:41:56.247327 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:56.247146    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="86fc9439-8986-4076-a3ca-230751d25f46" containerName="registry-server"
Jul 16 15:41:56.247327 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:56.247195    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="86fc9439-8986-4076-a3ca-230751d25f46" containerName="registry-server"
Jul 16 15:41:56.251242 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:56.251215    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9lgck"
Jul 16 15:41:56.253904 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:56.253881    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-qh4zv\""
Jul 16 15:41:56.259801 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:56.259778    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9lgck"]
Jul 16 15:41:56.367390 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:56.367357    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn7wp\" (UniqueName: \"kubernetes.io/projected/a7a0d14c-6f16-4d9a-929a-3695870b25e2-kube-api-access-nn7wp\") pod \"authorino-operator-75cc8d5cb7-9lgck\" (UID: \"a7a0d14c-6f16-4d9a-929a-3695870b25e2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9lgck"
Jul 16 15:41:56.468080 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:56.468046    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn7wp\" (UniqueName: \"kubernetes.io/projected/a7a0d14c-6f16-4d9a-929a-3695870b25e2-kube-api-access-nn7wp\") pod \"authorino-operator-75cc8d5cb7-9lgck\" (UID: \"a7a0d14c-6f16-4d9a-929a-3695870b25e2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9lgck"
Jul 16 15:41:56.481029 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:56.481005    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn7wp\" (UniqueName: \"kubernetes.io/projected/a7a0d14c-6f16-4d9a-929a-3695870b25e2-kube-api-access-nn7wp\") pod \"authorino-operator-75cc8d5cb7-9lgck\" (UID: \"a7a0d14c-6f16-4d9a-929a-3695870b25e2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9lgck"
Jul 16 15:41:56.561415 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:56.561328    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9lgck"
Jul 16 15:41:56.690288 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:56.690171    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9lgck"]
Jul 16 15:41:56.692967 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:41:56.692922    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7a0d14c_6f16_4d9a_929a_3695870b25e2.slice/crio-b7cf5bb1e20b9c3c1d88fb569b9bf26a99e4644cc2a24d4275541865c7a0a0e4 WatchSource:0}: Error finding container b7cf5bb1e20b9c3c1d88fb569b9bf26a99e4644cc2a24d4275541865c7a0a0e4: Status 404 returned error can't find the container with id b7cf5bb1e20b9c3c1d88fb569b9bf26a99e4644cc2a24d4275541865c7a0a0e4
Jul 16 15:41:56.763329 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:56.763295    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9lgck" event={"ID":"a7a0d14c-6f16-4d9a-929a-3695870b25e2","Type":"ContainerStarted","Data":"b7cf5bb1e20b9c3c1d88fb569b9bf26a99e4644cc2a24d4275541865c7a0a0e4"}
Jul 16 15:41:57.603473 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:57.603437    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:41:59.508840 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:59.508807    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-l24fv"]
Jul 16 15:41:59.512079 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:59.512059    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l24fv"
Jul 16 15:41:59.515137 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:59.515117    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 16 15:41:59.515652 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:59.515631    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-9snp2\""
Jul 16 15:41:59.524382 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:59.524359    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-l24fv"]
Jul 16 15:41:59.597343 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:59.597309    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mpsx\" (UniqueName: \"kubernetes.io/projected/46ec8fae-a977-4a40-94d9-a83ab2c75916-kube-api-access-5mpsx\") pod \"dns-operator-controller-manager-864dd78c4c-l24fv\" (UID: \"46ec8fae-a977-4a40-94d9-a83ab2c75916\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l24fv"
Jul 16 15:41:59.648155 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:59.648119    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:41:59.698616 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:59.698582    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mpsx\" (UniqueName: \"kubernetes.io/projected/46ec8fae-a977-4a40-94d9-a83ab2c75916-kube-api-access-5mpsx\") pod \"dns-operator-controller-manager-864dd78c4c-l24fv\" (UID: \"46ec8fae-a977-4a40-94d9-a83ab2c75916\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l24fv"
Jul 16 15:41:59.737797 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:59.737767    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mpsx\" (UniqueName: \"kubernetes.io/projected/46ec8fae-a977-4a40-94d9-a83ab2c75916-kube-api-access-5mpsx\") pod \"dns-operator-controller-manager-864dd78c4c-l24fv\" (UID: \"46ec8fae-a977-4a40-94d9-a83ab2c75916\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l24fv"
Jul 16 15:41:59.779632 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:59.779550    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9lgck" event={"ID":"a7a0d14c-6f16-4d9a-929a-3695870b25e2","Type":"ContainerStarted","Data":"dd22b01c06d4f246b31f62f2f92ebc958fd82dfdecf398bd1edcd27bf9952392"}
Jul 16 15:41:59.779765 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:59.779682    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9lgck"
Jul 16 15:41:59.822469 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:59.822443    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l24fv"
Jul 16 15:41:59.958463 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:59.958407    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9lgck" podStartSLOduration=1.691150527 podStartE2EDuration="3.95838157s" totalImagesPullingTime="2.267231043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:41:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:41:59.810249298 +0000 UTC m=+385.999113252" watchObservedRunningTime="2026-07-16 15:41:59.95838157 +0000 UTC m=+386.147245526"
Jul 16 15:41:59.959115 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:41:59.959096    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-l24fv"]
Jul 16 15:41:59.961411 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:41:59.961383    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46ec8fae_a977_4a40_94d9_a83ab2c75916.slice/crio-dd662ece4a8c0ea573e72a5c619971f696695aaca5f46241d849387acdd3cb4c WatchSource:0}: Error finding container dd662ece4a8c0ea573e72a5c619971f696695aaca5f46241d849387acdd3cb4c: Status 404 returned error can't find the container with id dd662ece4a8c0ea573e72a5c619971f696695aaca5f46241d849387acdd3cb4c
Jul 16 15:42:00.784426 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:00.784391    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l24fv" event={"ID":"46ec8fae-a977-4a40-94d9-a83ab2c75916","Type":"ContainerStarted","Data":"dd662ece4a8c0ea573e72a5c619971f696695aaca5f46241d849387acdd3cb4c"}
Jul 16 15:42:01.249018 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:01.248983    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:01.354662 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:01.354608    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:01.523120 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:01.523028    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:03.072944 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:03.072907    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"]
Jul 16 15:42:03.076602 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:03.076579    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:03.082339 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:03.082317    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-9bzrh\""
Jul 16 15:42:03.112163 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:03.112127    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"]
Jul 16 15:42:03.129305 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:03.129270    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krnkj\" (UniqueName: \"kubernetes.io/projected/55addc44-783b-430f-9f78-314cf6e0ac17-kube-api-access-krnkj\") pod \"limitador-operator-controller-manager-859d7b78bd-scpnf\" (UID: \"55addc44-783b-430f-9f78-314cf6e0ac17\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:03.230524 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:03.230487    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krnkj\" (UniqueName: \"kubernetes.io/projected/55addc44-783b-430f-9f78-314cf6e0ac17-kube-api-access-krnkj\") pod \"limitador-operator-controller-manager-859d7b78bd-scpnf\" (UID: \"55addc44-783b-430f-9f78-314cf6e0ac17\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:03.288945 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:03.288900    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krnkj\" (UniqueName: \"kubernetes.io/projected/55addc44-783b-430f-9f78-314cf6e0ac17-kube-api-access-krnkj\") pod \"limitador-operator-controller-manager-859d7b78bd-scpnf\" (UID: \"55addc44-783b-430f-9f78-314cf6e0ac17\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:03.391210 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:03.391158    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:03.567301 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:03.567264    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"]
Jul 16 15:42:03.572344 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:42:03.572311    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55addc44_783b_430f_9f78_314cf6e0ac17.slice/crio-1191f9f44b3f0556ccf2acbd6989110a189c595a1dd3d6fba61a5055bb97d3bf WatchSource:0}: Error finding container 1191f9f44b3f0556ccf2acbd6989110a189c595a1dd3d6fba61a5055bb97d3bf: Status 404 returned error can't find the container with id 1191f9f44b3f0556ccf2acbd6989110a189c595a1dd3d6fba61a5055bb97d3bf
Jul 16 15:42:03.688295 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:03.688192    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:03.798082 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:03.798034    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l24fv" event={"ID":"46ec8fae-a977-4a40-94d9-a83ab2c75916","Type":"ContainerStarted","Data":"c38fcfdcf303b660684a38df991c3de31396c4bd565704ddbbd54c09c6c5567f"}
Jul 16 15:42:03.798275 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:03.798106    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l24fv"
Jul 16 15:42:03.799154 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:03.799134    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf" event={"ID":"55addc44-783b-430f-9f78-314cf6e0ac17","Type":"ContainerStarted","Data":"1191f9f44b3f0556ccf2acbd6989110a189c595a1dd3d6fba61a5055bb97d3bf"}
Jul 16 15:42:03.837835 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:03.837775    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l24fv" podStartSLOduration=1.60430027 podStartE2EDuration="4.837756292s" totalImagesPullingTime="3.233456022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:42:03.836083213 +0000 UTC m=+390.024947191" watchObservedRunningTime="2026-07-16 15:42:03.837756292 +0000 UTC m=+390.026620247"
Jul 16 15:42:04.093197 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:04.093168    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:05.809645 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:05.809612    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf" event={"ID":"55addc44-783b-430f-9f78-314cf6e0ac17","Type":"ContainerStarted","Data":"11b14a42d53767e83a724d2c24a2ff25242a80606e31f20fac874719083cab3c"}
Jul 16 15:42:05.810157 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:05.809746    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:05.830805 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:05.830755    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf" podStartSLOduration=1.130576622 podStartE2EDuration="2.830740153s" totalImagesPullingTime="1.700163531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:42:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:42:05.830104411 +0000 UTC m=+392.018968379" watchObservedRunningTime="2026-07-16 15:42:05.830740153 +0000 UTC m=+392.019604160"
Jul 16 15:42:07.125282 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:07.125247    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:08.123482 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:08.123449    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"]
Jul 16 15:42:08.128593 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:08.127479    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:08.131890 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:08.131867    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nlqrf\""
Jul 16 15:42:08.146282 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:08.146257    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"]
Jul 16 15:42:08.273842 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:08.273808    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff0a7e16-005b-4dad-b450-aa44386c4d4c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" (UID: \"ff0a7e16-005b-4dad-b450-aa44386c4d4c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:08.274013 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:08.273848    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj9vt\" (UniqueName: \"kubernetes.io/projected/ff0a7e16-005b-4dad-b450-aa44386c4d4c-kube-api-access-jj9vt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" (UID: \"ff0a7e16-005b-4dad-b450-aa44386c4d4c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:08.375286 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:08.375209    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff0a7e16-005b-4dad-b450-aa44386c4d4c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" (UID: \"ff0a7e16-005b-4dad-b450-aa44386c4d4c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:08.375286 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:08.375247    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj9vt\" (UniqueName: \"kubernetes.io/projected/ff0a7e16-005b-4dad-b450-aa44386c4d4c-kube-api-access-jj9vt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" (UID: \"ff0a7e16-005b-4dad-b450-aa44386c4d4c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:08.375681 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:08.375650    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff0a7e16-005b-4dad-b450-aa44386c4d4c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" (UID: \"ff0a7e16-005b-4dad-b450-aa44386c4d4c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:08.384195 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:08.384174    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj9vt\" (UniqueName: \"kubernetes.io/projected/ff0a7e16-005b-4dad-b450-aa44386c4d4c-kube-api-access-jj9vt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" (UID: \"ff0a7e16-005b-4dad-b450-aa44386c4d4c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:08.439265 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:08.439240    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:08.581969 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:08.581945    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"]
Jul 16 15:42:08.584020 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:42:08.583986    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0a7e16_005b_4dad_b450_aa44386c4d4c.slice/crio-fbc2dd96bf0aeb3dfd2304367de78c43932865ae43e76de63a0beffecac02282 WatchSource:0}: Error finding container fbc2dd96bf0aeb3dfd2304367de78c43932865ae43e76de63a0beffecac02282: Status 404 returned error can't find the container with id fbc2dd96bf0aeb3dfd2304367de78c43932865ae43e76de63a0beffecac02282
Jul 16 15:42:08.823054 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:08.822955    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s" event={"ID":"ff0a7e16-005b-4dad-b450-aa44386c4d4c","Type":"ContainerStarted","Data":"fbc2dd96bf0aeb3dfd2304367de78c43932865ae43e76de63a0beffecac02282"}
Jul 16 15:42:09.394730 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:09.394678    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:09.496512 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:09.496475    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:09.790105 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:09.790013    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:09.889735 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:09.889701    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:10.787083 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:10.787051    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9lgck"
Jul 16 15:42:13.399361 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:13.399326    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:13.463324 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:13.463289    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:14.025242 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:14.025213    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:14.083413 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:14.083370    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:14.806330 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:14.806302    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l24fv"
Jul 16 15:42:14.851157 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:14.851118    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s" event={"ID":"ff0a7e16-005b-4dad-b450-aa44386c4d4c","Type":"ContainerStarted","Data":"c118d6ecbdd53f6011e7a086174a44adf633fd550c37a31e4174ef5faf5e3c3d"}
Jul 16 15:42:14.851350 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:14.851242    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:14.899820 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:14.899762    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s" podStartSLOduration=1.39300983 podStartE2EDuration="6.899743312s" totalImagesPullingTime="5.506733482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:42:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:42:14.898429802 +0000 UTC m=+401.087293756" watchObservedRunningTime="2026-07-16 15:42:14.899743312 +0000 UTC m=+401.088607267"
Jul 16 15:42:16.816111 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:16.816074    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:17.047178 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.047145    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:17.070867 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.070788    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"]
Jul 16 15:42:17.074279 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.074263    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"
Jul 16 15:42:17.080331 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.080290    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 16 15:42:17.080485 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.080464    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4n4mw\""
Jul 16 15:42:17.081013 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.080998    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 16 15:42:17.097980 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.097954    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"]
Jul 16 15:42:17.150151 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.150120    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3fd0bb7-775a-456d-83bf-4af1b7d8c437-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vnp4r\" (UID: \"e3fd0bb7-775a-456d-83bf-4af1b7d8c437\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"
Jul 16 15:42:17.150151 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.150152    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e3fd0bb7-775a-456d-83bf-4af1b7d8c437-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vnp4r\" (UID: \"e3fd0bb7-775a-456d-83bf-4af1b7d8c437\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"
Jul 16 15:42:17.150374 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.150176    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6h29\" (UniqueName: \"kubernetes.io/projected/e3fd0bb7-775a-456d-83bf-4af1b7d8c437-kube-api-access-f6h29\") pod \"kuadrant-console-plugin-66b5774985-vnp4r\" (UID: \"e3fd0bb7-775a-456d-83bf-4af1b7d8c437\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"
Jul 16 15:42:17.250799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.250768    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3fd0bb7-775a-456d-83bf-4af1b7d8c437-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vnp4r\" (UID: \"e3fd0bb7-775a-456d-83bf-4af1b7d8c437\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"
Jul 16 15:42:17.250799 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.250800    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e3fd0bb7-775a-456d-83bf-4af1b7d8c437-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vnp4r\" (UID: \"e3fd0bb7-775a-456d-83bf-4af1b7d8c437\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"
Jul 16 15:42:17.251026 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.250825    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6h29\" (UniqueName: \"kubernetes.io/projected/e3fd0bb7-775a-456d-83bf-4af1b7d8c437-kube-api-access-f6h29\") pod \"kuadrant-console-plugin-66b5774985-vnp4r\" (UID: \"e3fd0bb7-775a-456d-83bf-4af1b7d8c437\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"
Jul 16 15:42:17.251026 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:17.250958    2570 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 16 15:42:17.251088 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:17.251038    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3fd0bb7-775a-456d-83bf-4af1b7d8c437-plugin-serving-cert podName:e3fd0bb7-775a-456d-83bf-4af1b7d8c437 nodeName:}" failed. No retries permitted until 2026-07-16 15:42:17.751016445 +0000 UTC m=+403.939880382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/e3fd0bb7-775a-456d-83bf-4af1b7d8c437-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-vnp4r" (UID: "e3fd0bb7-775a-456d-83bf-4af1b7d8c437") : secret "plugin-serving-cert" not found
Jul 16 15:42:17.251445 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.251424    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e3fd0bb7-775a-456d-83bf-4af1b7d8c437-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vnp4r\" (UID: \"e3fd0bb7-775a-456d-83bf-4af1b7d8c437\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"
Jul 16 15:42:17.273742 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.273715    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6h29\" (UniqueName: \"kubernetes.io/projected/e3fd0bb7-775a-456d-83bf-4af1b7d8c437-kube-api-access-f6h29\") pod \"kuadrant-console-plugin-66b5774985-vnp4r\" (UID: \"e3fd0bb7-775a-456d-83bf-4af1b7d8c437\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"
Jul 16 15:42:17.484178 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.484142    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:17.635156 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.635122    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:17.756346 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.756262    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3fd0bb7-775a-456d-83bf-4af1b7d8c437-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vnp4r\" (UID: \"e3fd0bb7-775a-456d-83bf-4af1b7d8c437\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"
Jul 16 15:42:17.758591 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.758573    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3fd0bb7-775a-456d-83bf-4af1b7d8c437-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vnp4r\" (UID: \"e3fd0bb7-775a-456d-83bf-4af1b7d8c437\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"
Jul 16 15:42:17.983739 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:17.983710    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"
Jul 16 15:42:18.133111 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:18.133078    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r"]
Jul 16 15:42:18.137040 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:42:18.137013    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3fd0bb7_775a_456d_83bf_4af1b7d8c437.slice/crio-6c9aa97866da65c9cafd6c3440f7cc788b46e42a3b5241f1ce19727845f4adc7 WatchSource:0}: Error finding container 6c9aa97866da65c9cafd6c3440f7cc788b46e42a3b5241f1ce19727845f4adc7: Status 404 returned error can't find the container with id 6c9aa97866da65c9cafd6c3440f7cc788b46e42a3b5241f1ce19727845f4adc7
Jul 16 15:42:18.875239 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:18.875204    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r" event={"ID":"e3fd0bb7-775a-456d-83bf-4af1b7d8c437","Type":"ContainerStarted","Data":"6c9aa97866da65c9cafd6c3440f7cc788b46e42a3b5241f1ce19727845f4adc7"}
Jul 16 15:42:20.557680 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:20.557639    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:20.657805 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:20.657766    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:20.943822 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:20.943779    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:21.060987 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:21.060850    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:25.486083 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:25.486043    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:25.588160 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:25.588123    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:25.690419 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:25.690387    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:25.857857 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:25.857830    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:27.867618 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:27.863859    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"]
Jul 16 15:42:27.867618 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:27.864189    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s" podUID="ff0a7e16-005b-4dad-b450-aa44386c4d4c" containerName="manager" containerID="cri-o://c118d6ecbdd53f6011e7a086174a44adf633fd550c37a31e4174ef5faf5e3c3d" gracePeriod=2
Jul 16 15:42:27.893718 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:27.893684    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"]
Jul 16 15:42:27.933857 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:27.933821    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"]
Jul 16 15:42:27.934106 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:27.934075    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf" podUID="55addc44-783b-430f-9f78-314cf6e0ac17" containerName="manager" containerID="cri-o://11b14a42d53767e83a724d2c24a2ff25242a80606e31f20fac874719083cab3c" gracePeriod=2
Jul 16 15:42:27.942733 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:27.942696    2570 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ff0a7e16-005b-4dad-b450-aa44386c4d4c\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-pt44s\": pods \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:27.945270 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:27.945244    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"]
Jul 16 15:42:27.945270 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:27.945241    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-scpnf\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="55addc44-783b-430f-9f78-314cf6e0ac17" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:27.947418 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:27.947387    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="ff0a7e16-005b-4dad-b450-aa44386c4d4c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:27.984312 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:27.984274    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzk4t"]
Jul 16 15:42:27.984729 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:27.984670    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff0a7e16-005b-4dad-b450-aa44386c4d4c" containerName="manager"
Jul 16 15:42:27.984729 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:27.984691    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff0a7e16-005b-4dad-b450-aa44386c4d4c" containerName="manager"
Jul 16 15:42:27.984729 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:27.984708    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55addc44-783b-430f-9f78-314cf6e0ac17" containerName="manager"
Jul 16 15:42:27.984729 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:27.984716    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="55addc44-783b-430f-9f78-314cf6e0ac17" containerName="manager"
Jul 16 15:42:27.985127 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:27.984812    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff0a7e16-005b-4dad-b450-aa44386c4d4c" containerName="manager"
Jul 16 15:42:27.985127 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:27.984826    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="55addc44-783b-430f-9f78-314cf6e0ac17" containerName="manager"
Jul 16 15:42:27.990825 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:27.990788    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzk4t"
Jul 16 15:42:28.000385 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.000358    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"]
Jul 16 15:42:28.005524 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.005502    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"
Jul 16 15:42:28.007181 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:28.007139    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-scpnf\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="55addc44-783b-430f-9f78-314cf6e0ac17" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:28.052414 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.052380    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb09c330-c3c0-4c37-9508-5def26d2f1a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6g8cx\" (UID: \"eb09c330-c3c0-4c37-9508-5def26d2f1a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"
Jul 16 15:42:28.052595 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.052448    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9xpl\" (UniqueName: \"kubernetes.io/projected/24eda07a-6e1a-4f20-8fe9-f7815cc639f1-kube-api-access-q9xpl\") pod \"limitador-operator-controller-manager-859d7b78bd-vzk4t\" (UID: \"24eda07a-6e1a-4f20-8fe9-f7815cc639f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzk4t"
Jul 16 15:42:28.052595 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.052537    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qrb9\" (UniqueName: \"kubernetes.io/projected/eb09c330-c3c0-4c37-9508-5def26d2f1a3-kube-api-access-8qrb9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6g8cx\" (UID: \"eb09c330-c3c0-4c37-9508-5def26d2f1a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"
Jul 16 15:42:28.065373 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.065342    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzk4t"]
Jul 16 15:42:28.078231 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.078201    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"]
Jul 16 15:42:28.153193 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.153113    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9xpl\" (UniqueName: \"kubernetes.io/projected/24eda07a-6e1a-4f20-8fe9-f7815cc639f1-kube-api-access-q9xpl\") pod \"limitador-operator-controller-manager-859d7b78bd-vzk4t\" (UID: \"24eda07a-6e1a-4f20-8fe9-f7815cc639f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzk4t"
Jul 16 15:42:28.153193 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.153176    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qrb9\" (UniqueName: \"kubernetes.io/projected/eb09c330-c3c0-4c37-9508-5def26d2f1a3-kube-api-access-8qrb9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6g8cx\" (UID: \"eb09c330-c3c0-4c37-9508-5def26d2f1a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"
Jul 16 15:42:28.153414 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.153224    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb09c330-c3c0-4c37-9508-5def26d2f1a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6g8cx\" (UID: \"eb09c330-c3c0-4c37-9508-5def26d2f1a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"
Jul 16 15:42:28.153567 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.153551    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb09c330-c3c0-4c37-9508-5def26d2f1a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6g8cx\" (UID: \"eb09c330-c3c0-4c37-9508-5def26d2f1a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"
Jul 16 15:42:28.162451 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:28.162410    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="ff0a7e16-005b-4dad-b450-aa44386c4d4c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:28.185029 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.184999    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qrb9\" (UniqueName: \"kubernetes.io/projected/eb09c330-c3c0-4c37-9508-5def26d2f1a3-kube-api-access-8qrb9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6g8cx\" (UID: \"eb09c330-c3c0-4c37-9508-5def26d2f1a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"
Jul 16 15:42:28.188112 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:28.188082    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-scpnf\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="55addc44-783b-430f-9f78-314cf6e0ac17" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:28.195124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.195097    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9xpl\" (UniqueName: \"kubernetes.io/projected/24eda07a-6e1a-4f20-8fe9-f7815cc639f1-kube-api-access-q9xpl\") pod \"limitador-operator-controller-manager-859d7b78bd-vzk4t\" (UID: \"24eda07a-6e1a-4f20-8fe9-f7815cc639f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzk4t"
Jul 16 15:42:28.200042 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:28.199995    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="ff0a7e16-005b-4dad-b450-aa44386c4d4c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:28.318870 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.318837    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzk4t"
Jul 16 15:42:28.327232 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:28.327200    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"
Jul 16 15:42:29.143204 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:29.143156    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:29.284044 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:29.283999    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:29.697768 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:29.697719    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:29.811423 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:29.811380    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:33.744001 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:33.743964    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:33.873111 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:33.873080    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:34.846390 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:34.846343    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:34.947031 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:34.946993    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:37.221007 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:37.220962    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:37.318344 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:37.318307    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:40.349631 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.349583    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:40.447696 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.447662    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:40.534459 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.534408    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:42:40.638753 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.638355    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzk4t"]
Jul 16 15:42:40.639732 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:42:40.639694    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24eda07a_6e1a_4f20_8fe9_f7815cc639f1.slice/crio-8d3f6c8dd2bfc43c9a4283ae0ce0688f2bd9a128fb412882f79252fbc267d987 WatchSource:0}: Error finding container 8d3f6c8dd2bfc43c9a4283ae0ce0688f2bd9a128fb412882f79252fbc267d987: Status 404 returned error can't find the container with id 8d3f6c8dd2bfc43c9a4283ae0ce0688f2bd9a128fb412882f79252fbc267d987
Jul 16 15:42:40.648076 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.648050    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:40.650429 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:40.650396    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="ff0a7e16-005b-4dad-b450-aa44386c4d4c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:40.655163 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.655143    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:40.657191 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:40.657158    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-scpnf\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="55addc44-783b-430f-9f78-314cf6e0ac17" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:40.659010 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:40.658975    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="ff0a7e16-005b-4dad-b450-aa44386c4d4c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:40.663227 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.663208    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"]
Jul 16 15:42:40.673028 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:42:40.673000    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb09c330_c3c0_4c37_9508_5def26d2f1a3.slice/crio-706376ac58d4b6afbf7d3875dd757d0082691b7532689a9e4517fee63dcba99b WatchSource:0}: Error finding container 706376ac58d4b6afbf7d3875dd757d0082691b7532689a9e4517fee63dcba99b: Status 404 returned error can't find the container with id 706376ac58d4b6afbf7d3875dd757d0082691b7532689a9e4517fee63dcba99b
Jul 16 15:42:40.769374 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.769346    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj9vt\" (UniqueName: \"kubernetes.io/projected/ff0a7e16-005b-4dad-b450-aa44386c4d4c-kube-api-access-jj9vt\") pod \"ff0a7e16-005b-4dad-b450-aa44386c4d4c\" (UID: \"ff0a7e16-005b-4dad-b450-aa44386c4d4c\") "
Jul 16 15:42:40.769565 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.769420    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krnkj\" (UniqueName: \"kubernetes.io/projected/55addc44-783b-430f-9f78-314cf6e0ac17-kube-api-access-krnkj\") pod \"55addc44-783b-430f-9f78-314cf6e0ac17\" (UID: \"55addc44-783b-430f-9f78-314cf6e0ac17\") "
Jul 16 15:42:40.769565 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.769442    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff0a7e16-005b-4dad-b450-aa44386c4d4c-extensions-socket-volume\") pod \"ff0a7e16-005b-4dad-b450-aa44386c4d4c\" (UID: \"ff0a7e16-005b-4dad-b450-aa44386c4d4c\") "
Jul 16 15:42:40.769966 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.769917    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff0a7e16-005b-4dad-b450-aa44386c4d4c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ff0a7e16-005b-4dad-b450-aa44386c4d4c" (UID: "ff0a7e16-005b-4dad-b450-aa44386c4d4c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 15:42:40.771430 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.771405    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff0a7e16-005b-4dad-b450-aa44386c4d4c-kube-api-access-jj9vt" (OuterVolumeSpecName: "kube-api-access-jj9vt") pod "ff0a7e16-005b-4dad-b450-aa44386c4d4c" (UID: "ff0a7e16-005b-4dad-b450-aa44386c4d4c"). InnerVolumeSpecName "kube-api-access-jj9vt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:42:40.771510 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.771435    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55addc44-783b-430f-9f78-314cf6e0ac17-kube-api-access-krnkj" (OuterVolumeSpecName: "kube-api-access-krnkj") pod "55addc44-783b-430f-9f78-314cf6e0ac17" (UID: "55addc44-783b-430f-9f78-314cf6e0ac17"). InnerVolumeSpecName "kube-api-access-krnkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:42:40.870293 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.870212    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj9vt\" (UniqueName: \"kubernetes.io/projected/ff0a7e16-005b-4dad-b450-aa44386c4d4c-kube-api-access-jj9vt\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:42:40.870293 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.870238    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krnkj\" (UniqueName: \"kubernetes.io/projected/55addc44-783b-430f-9f78-314cf6e0ac17-kube-api-access-krnkj\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:42:40.870293 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.870248    2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff0a7e16-005b-4dad-b450-aa44386c4d4c-extensions-socket-volume\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:42:40.991183 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.991151    2570 generic.go:356] "Generic (PLEG): container finished" podID="ff0a7e16-005b-4dad-b450-aa44386c4d4c" containerID="c118d6ecbdd53f6011e7a086174a44adf633fd550c37a31e4174ef5faf5e3c3d" exitCode=0
Jul 16 15:42:40.991366 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.991190    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:40.991366 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.991248    2570 scope.go:117] "RemoveContainer" containerID="c118d6ecbdd53f6011e7a086174a44adf633fd550c37a31e4174ef5faf5e3c3d"
Jul 16 15:42:40.992833 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.992792    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzk4t" event={"ID":"24eda07a-6e1a-4f20-8fe9-f7815cc639f1","Type":"ContainerStarted","Data":"13fa969df428a7306d717e4f9a779fd90e84b96894fa04184f3ac700d67f7090"}
Jul 16 15:42:40.992969 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.992843    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzk4t" event={"ID":"24eda07a-6e1a-4f20-8fe9-f7815cc639f1","Type":"ContainerStarted","Data":"8d3f6c8dd2bfc43c9a4283ae0ce0688f2bd9a128fb412882f79252fbc267d987"}
Jul 16 15:42:40.992969 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.992901    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzk4t"
Jul 16 15:42:40.993632 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:40.993609    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-scpnf\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="55addc44-783b-430f-9f78-314cf6e0ac17" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:40.994145 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.994122    2570 generic.go:356] "Generic (PLEG): container finished" podID="55addc44-783b-430f-9f78-314cf6e0ac17" containerID="11b14a42d53767e83a724d2c24a2ff25242a80606e31f20fac874719083cab3c" exitCode=0
Jul 16 15:42:40.994215 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.994165    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:40.995764 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.995742    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx" event={"ID":"eb09c330-c3c0-4c37-9508-5def26d2f1a3","Type":"ContainerStarted","Data":"ebd606eb614597e2c88416a48d74b849fb7a4edf21def763b44ce0d4f643120d"}
Jul 16 15:42:40.995852 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.995783    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx" event={"ID":"eb09c330-c3c0-4c37-9508-5def26d2f1a3","Type":"ContainerStarted","Data":"706376ac58d4b6afbf7d3875dd757d0082691b7532689a9e4517fee63dcba99b"}
Jul 16 15:42:40.995852 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.995806    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"
Jul 16 15:42:40.995959 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:40.995871    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="ff0a7e16-005b-4dad-b450-aa44386c4d4c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:40.997434 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:40.997417    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r" event={"ID":"e3fd0bb7-775a-456d-83bf-4af1b7d8c437","Type":"ContainerStarted","Data":"396a488863ea18a961f2937db6c174717ccf50359d834adf0e0c79e6ec75e28c"}
Jul 16 15:42:41.001205 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:41.001174    2570 scope.go:117] "RemoveContainer" containerID="c118d6ecbdd53f6011e7a086174a44adf633fd550c37a31e4174ef5faf5e3c3d"
Jul 16 15:42:41.001491 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:41.001454    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c118d6ecbdd53f6011e7a086174a44adf633fd550c37a31e4174ef5faf5e3c3d\": container with ID starting with c118d6ecbdd53f6011e7a086174a44adf633fd550c37a31e4174ef5faf5e3c3d not found: ID does not exist" containerID="c118d6ecbdd53f6011e7a086174a44adf633fd550c37a31e4174ef5faf5e3c3d"
Jul 16 15:42:41.001571 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:41.001497    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c118d6ecbdd53f6011e7a086174a44adf633fd550c37a31e4174ef5faf5e3c3d"} err="failed to get container status \"c118d6ecbdd53f6011e7a086174a44adf633fd550c37a31e4174ef5faf5e3c3d\": rpc error: code = NotFound desc = could not find container \"c118d6ecbdd53f6011e7a086174a44adf633fd550c37a31e4174ef5faf5e3c3d\": container with ID starting with c118d6ecbdd53f6011e7a086174a44adf633fd550c37a31e4174ef5faf5e3c3d not found: ID does not exist"
Jul 16 15:42:41.001571 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:41.001514    2570 scope.go:117] "RemoveContainer" containerID="11b14a42d53767e83a724d2c24a2ff25242a80606e31f20fac874719083cab3c"
Jul 16 15:42:41.008693 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:41.008673    2570 scope.go:117] "RemoveContainer" containerID="11b14a42d53767e83a724d2c24a2ff25242a80606e31f20fac874719083cab3c"
Jul 16 15:42:41.010317 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:41.009218    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11b14a42d53767e83a724d2c24a2ff25242a80606e31f20fac874719083cab3c\": container with ID starting with 11b14a42d53767e83a724d2c24a2ff25242a80606e31f20fac874719083cab3c not found: ID does not exist" containerID="11b14a42d53767e83a724d2c24a2ff25242a80606e31f20fac874719083cab3c"
Jul 16 15:42:41.010317 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:41.009248    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11b14a42d53767e83a724d2c24a2ff25242a80606e31f20fac874719083cab3c"} err="failed to get container status \"11b14a42d53767e83a724d2c24a2ff25242a80606e31f20fac874719083cab3c\": rpc error: code = NotFound desc = could not find container \"11b14a42d53767e83a724d2c24a2ff25242a80606e31f20fac874719083cab3c\": container with ID starting with 11b14a42d53767e83a724d2c24a2ff25242a80606e31f20fac874719083cab3c not found: ID does not exist"
Jul 16 15:42:41.019153 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:41.019114    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx" podStartSLOduration=14.019100906 podStartE2EDuration="14.019100906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:42:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:42:41.017332609 +0000 UTC m=+427.206196565" watchObservedRunningTime="2026-07-16 15:42:41.019100906 +0000 UTC m=+427.207964839"
Jul 16 15:42:41.019506 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:41.019474    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-scpnf\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="55addc44-783b-430f-9f78-314cf6e0ac17" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:41.043222 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:41.043178    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzk4t" podStartSLOduration=14.043165651 podStartE2EDuration="14.043165651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:42:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:42:41.042654425 +0000 UTC m=+427.231518403" watchObservedRunningTime="2026-07-16 15:42:41.043165651 +0000 UTC m=+427.232029606"
Jul 16 15:42:41.044744 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:41.044717    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="ff0a7e16-005b-4dad-b450-aa44386c4d4c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:41.046602 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:41.046580    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-scpnf\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="55addc44-783b-430f-9f78-314cf6e0ac17" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-scpnf"
Jul 16 15:42:41.048332 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:41.048313    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pt44s\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="ff0a7e16-005b-4dad-b450-aa44386c4d4c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pt44s"
Jul 16 15:42:41.066022 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:41.065981    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vnp4r" podStartSLOduration=1.63102419 podStartE2EDuration="24.065969338s" totalImagesPullingTime="22.434945148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:42:41.064294823 +0000 UTC m=+427.253158777" watchObservedRunningTime="2026-07-16 15:42:41.065969338 +0000 UTC m=+427.254833293"
Jul 16 15:42:42.266749 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:42.266718    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55addc44-783b-430f-9f78-314cf6e0ac17" path="/var/lib/kubelet/pods/55addc44-783b-430f-9f78-314cf6e0ac17/volumes"
Jul 16 15:42:42.267129 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:42.267066    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff0a7e16-005b-4dad-b450-aa44386c4d4c" path="/var/lib/kubelet/pods/ff0a7e16-005b-4dad-b450-aa44386c4d4c/volumes"
Jul 16 15:42:47.409761 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:47.409725    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"]
Jul 16 15:42:47.410222 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:47.409895    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx" podUID="eb09c330-c3c0-4c37-9508-5def26d2f1a3" containerName="manager" containerID="cri-o://ebd606eb614597e2c88416a48d74b849fb7a4edf21def763b44ce0d4f643120d" gracePeriod=10
Jul 16 15:42:47.411834 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:47.411804    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"
Jul 16 15:42:47.660776 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:47.660716    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"
Jul 16 15:42:47.725085 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:47.725055    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qrb9\" (UniqueName: \"kubernetes.io/projected/eb09c330-c3c0-4c37-9508-5def26d2f1a3-kube-api-access-8qrb9\") pod \"eb09c330-c3c0-4c37-9508-5def26d2f1a3\" (UID: \"eb09c330-c3c0-4c37-9508-5def26d2f1a3\") "
Jul 16 15:42:47.725242 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:47.725095    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb09c330-c3c0-4c37-9508-5def26d2f1a3-extensions-socket-volume\") pod \"eb09c330-c3c0-4c37-9508-5def26d2f1a3\" (UID: \"eb09c330-c3c0-4c37-9508-5def26d2f1a3\") "
Jul 16 15:42:47.725535 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:47.725514    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb09c330-c3c0-4c37-9508-5def26d2f1a3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "eb09c330-c3c0-4c37-9508-5def26d2f1a3" (UID: "eb09c330-c3c0-4c37-9508-5def26d2f1a3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 15:42:47.727316 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:47.727286    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb09c330-c3c0-4c37-9508-5def26d2f1a3-kube-api-access-8qrb9" (OuterVolumeSpecName: "kube-api-access-8qrb9") pod "eb09c330-c3c0-4c37-9508-5def26d2f1a3" (UID: "eb09c330-c3c0-4c37-9508-5def26d2f1a3"). InnerVolumeSpecName "kube-api-access-8qrb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:42:47.826714 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:47.826667    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qrb9\" (UniqueName: \"kubernetes.io/projected/eb09c330-c3c0-4c37-9508-5def26d2f1a3-kube-api-access-8qrb9\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:42:47.826714 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:47.826707    2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb09c330-c3c0-4c37-9508-5def26d2f1a3-extensions-socket-volume\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:42:48.035664 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:48.035579    2570 generic.go:356] "Generic (PLEG): container finished" podID="eb09c330-c3c0-4c37-9508-5def26d2f1a3" containerID="ebd606eb614597e2c88416a48d74b849fb7a4edf21def763b44ce0d4f643120d" exitCode=0
Jul 16 15:42:48.035664 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:48.035622    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"
Jul 16 15:42:48.035839 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:48.035667    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx" event={"ID":"eb09c330-c3c0-4c37-9508-5def26d2f1a3","Type":"ContainerDied","Data":"ebd606eb614597e2c88416a48d74b849fb7a4edf21def763b44ce0d4f643120d"}
Jul 16 15:42:48.035839 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:48.035719    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx" event={"ID":"eb09c330-c3c0-4c37-9508-5def26d2f1a3","Type":"ContainerDied","Data":"706376ac58d4b6afbf7d3875dd757d0082691b7532689a9e4517fee63dcba99b"}
Jul 16 15:42:48.035839 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:48.035742    2570 scope.go:117] "RemoveContainer" containerID="ebd606eb614597e2c88416a48d74b849fb7a4edf21def763b44ce0d4f643120d"
Jul 16 15:42:48.044717 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:48.044698    2570 scope.go:117] "RemoveContainer" containerID="ebd606eb614597e2c88416a48d74b849fb7a4edf21def763b44ce0d4f643120d"
Jul 16 15:42:48.045050 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:42:48.045029    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebd606eb614597e2c88416a48d74b849fb7a4edf21def763b44ce0d4f643120d\": container with ID starting with ebd606eb614597e2c88416a48d74b849fb7a4edf21def763b44ce0d4f643120d not found: ID does not exist" containerID="ebd606eb614597e2c88416a48d74b849fb7a4edf21def763b44ce0d4f643120d"
Jul 16 15:42:48.045095 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:48.045059    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebd606eb614597e2c88416a48d74b849fb7a4edf21def763b44ce0d4f643120d"} err="failed to get container status \"ebd606eb614597e2c88416a48d74b849fb7a4edf21def763b44ce0d4f643120d\": rpc error: code = NotFound desc = could not find container \"ebd606eb614597e2c88416a48d74b849fb7a4edf21def763b44ce0d4f643120d\": container with ID starting with ebd606eb614597e2c88416a48d74b849fb7a4edf21def763b44ce0d4f643120d not found: ID does not exist"
Jul 16 15:42:48.057081 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:48.057046    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"]
Jul 16 15:42:48.063568 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:48.063541    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6g8cx"]
Jul 16 15:42:48.267119 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:48.267091    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb09c330-c3c0-4c37-9508-5def26d2f1a3" path="/var/lib/kubelet/pods/eb09c330-c3c0-4c37-9508-5def26d2f1a3/volumes"
Jul 16 15:42:52.006361 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:42:52.006327    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzk4t"
Jul 16 15:43:03.812529 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:03.812449    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"]
Jul 16 15:43:03.812943 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:03.812919    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb09c330-c3c0-4c37-9508-5def26d2f1a3" containerName="manager"
Jul 16 15:43:03.813039 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:03.812959    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb09c330-c3c0-4c37-9508-5def26d2f1a3" containerName="manager"
Jul 16 15:43:03.813089 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:03.813044    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb09c330-c3c0-4c37-9508-5def26d2f1a3" containerName="manager"
Jul 16 15:43:03.835814 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:03.835776    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:03.919995 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:03.919958    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"]
Jul 16 15:43:03.920174 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:03.920013    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:03.921352 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:03.921301    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:03.922789 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:03.922769    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-lsw9r\""
Jul 16 15:43:04.071630 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.071536    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.071630 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.071583    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.071854 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.071672    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgr2z\" (UniqueName: \"kubernetes.io/projected/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-kube-api-access-zgr2z\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.071854 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.071747    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.071854 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.071782    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.071854 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.071801    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.071854 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.071839    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.072135 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.071876    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.072135 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.071912    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.173195 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.173161    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.173195 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.173200    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.173384 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.173220    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.173384 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.173345    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.173384 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.173377    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.173498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.173403    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.173498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.173439    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.173498 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.173458    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.173681 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.173520    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgr2z\" (UniqueName: \"kubernetes.io/projected/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-kube-api-access-zgr2z\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.173681 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.173632    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.173774 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.173702    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.173827 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.173774    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.173884 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.173826    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.174314 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.174295    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.175385 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.175365    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.175949 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.175913    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.181147 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.181123    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.181912 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.181896    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgr2z\" (UniqueName: \"kubernetes.io/projected/ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c-kube-api-access-zgr2z\") pod \"maas-default-gateway-openshift-default-56654dcf7-rcsbl\" (UID: \"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.230615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.230578    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:04.399949 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.399865    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"]
Jul 16 15:43:04.402562 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:43:04.402531    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec8e2873_e0cc_4395_b2c9_2ae1f99c4c1c.slice/crio-ee52d9a2ea387d3fc58ef58eac428e79c0c20c7992b74288d403e4164ed6c99f WatchSource:0}: Error finding container ee52d9a2ea387d3fc58ef58eac428e79c0c20c7992b74288d403e4164ed6c99f: Status 404 returned error can't find the container with id ee52d9a2ea387d3fc58ef58eac428e79c0c20c7992b74288d403e4164ed6c99f
Jul 16 15:43:04.404577 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.404550    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 15:43:04.404649 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.404620    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 15:43:04.404691 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:04.404649    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 15:43:05.111301 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:05.111258    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl" event={"ID":"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c","Type":"ContainerStarted","Data":"8fbd2348e2ce9d5d0cbdc687850db16c3a03b51906363cdb88e0ab72805c9cc8"}
Jul 16 15:43:05.111301 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:05.111306    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl" event={"ID":"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c","Type":"ContainerStarted","Data":"ee52d9a2ea387d3fc58ef58eac428e79c0c20c7992b74288d403e4164ed6c99f"}
Jul 16 15:43:05.132130 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:05.132083    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl" podStartSLOduration=2.132064777 podStartE2EDuration="2.132064777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:43:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:43:05.130766813 +0000 UTC m=+451.319630780" watchObservedRunningTime="2026-07-16 15:43:05.132064777 +0000 UTC m=+451.320928732"
Jul 16 15:43:05.231533 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:05.231505    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:05.236403 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:05.236382    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:06.115434 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.115408    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:06.116411 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.116389    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:43:06.153655 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.153625    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r245n"]
Jul 16 15:43:06.298754 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.298719    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:06.399240 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.399161    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:06.648887 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.648856    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n"
Jul 16 15:43:06.651982 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.651897    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 16 15:43:06.656788 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.656758    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r245n"]
Jul 16 15:43:06.656906 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.656827    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r245n"]
Jul 16 15:43:06.695301 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.695270    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpt7t\" (UniqueName: \"kubernetes.io/projected/6e52fab9-13a6-4944-aaa9-38ec79c8873b-kube-api-access-dpt7t\") pod \"limitador-limitador-64c94fd64f-r245n\" (UID: \"6e52fab9-13a6-4944-aaa9-38ec79c8873b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n"
Jul 16 15:43:06.695445 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.695322    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6e52fab9-13a6-4944-aaa9-38ec79c8873b-config-file\") pod \"limitador-limitador-64c94fd64f-r245n\" (UID: \"6e52fab9-13a6-4944-aaa9-38ec79c8873b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n"
Jul 16 15:43:06.796050 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.796018    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6e52fab9-13a6-4944-aaa9-38ec79c8873b-config-file\") pod \"limitador-limitador-64c94fd64f-r245n\" (UID: \"6e52fab9-13a6-4944-aaa9-38ec79c8873b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n"
Jul 16 15:43:06.796229 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.796124    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpt7t\" (UniqueName: \"kubernetes.io/projected/6e52fab9-13a6-4944-aaa9-38ec79c8873b-kube-api-access-dpt7t\") pod \"limitador-limitador-64c94fd64f-r245n\" (UID: \"6e52fab9-13a6-4944-aaa9-38ec79c8873b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n"
Jul 16 15:43:06.796691 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.796672    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6e52fab9-13a6-4944-aaa9-38ec79c8873b-config-file\") pod \"limitador-limitador-64c94fd64f-r245n\" (UID: \"6e52fab9-13a6-4944-aaa9-38ec79c8873b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n"
Jul 16 15:43:06.804447 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.804424    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpt7t\" (UniqueName: \"kubernetes.io/projected/6e52fab9-13a6-4944-aaa9-38ec79c8873b-kube-api-access-dpt7t\") pod \"limitador-limitador-64c94fd64f-r245n\" (UID: \"6e52fab9-13a6-4944-aaa9-38ec79c8873b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n"
Jul 16 15:43:06.963531 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.963450    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n"
Jul 16 15:43:06.968921 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.968896    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-rblf5"]
Jul 16 15:43:06.998490 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:06.998459    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:07.036420 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.036405    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rblf5"]
Jul 16 15:43:07.036497 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.036486    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rblf5"
Jul 16 15:43:07.039083 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.039063    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lg7h5\""
Jul 16 15:43:07.057862 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.057823    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:07.098609 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.098583    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbrdd\" (UniqueName: \"kubernetes.io/projected/f9f56ca9-643a-4da8-8608-1621ea57dced-kube-api-access-rbrdd\") pod \"authorino-58cd955675-rblf5\" (UID: \"f9f56ca9-643a-4da8-8608-1621ea57dced\") " pod="kuadrant-system/authorino-58cd955675-rblf5"
Jul 16 15:43:07.104679 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.104654    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-v9ph6"]
Jul 16 15:43:07.110228 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.110207    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-v9ph6"
Jul 16 15:43:07.112844 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.112824    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v9ph6"]
Jul 16 15:43:07.169596 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.169562    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r245n"]
Jul 16 15:43:07.173483 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:43:07.173458    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e52fab9_13a6_4944_aaa9_38ec79c8873b.slice/crio-e1e21e47f18178d59fde034a0ff5639a1300e05a283db7eb5c7acb55e622393a WatchSource:0}: Error finding container e1e21e47f18178d59fde034a0ff5639a1300e05a283db7eb5c7acb55e622393a: Status 404 returned error can't find the container with id e1e21e47f18178d59fde034a0ff5639a1300e05a283db7eb5c7acb55e622393a
Jul 16 15:43:07.199483 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.199455    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbrdd\" (UniqueName: \"kubernetes.io/projected/f9f56ca9-643a-4da8-8608-1621ea57dced-kube-api-access-rbrdd\") pod \"authorino-58cd955675-rblf5\" (UID: \"f9f56ca9-643a-4da8-8608-1621ea57dced\") " pod="kuadrant-system/authorino-58cd955675-rblf5"
Jul 16 15:43:07.199596 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.199502    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wdwz\" (UniqueName: \"kubernetes.io/projected/9a64b673-c726-44d1-af6c-65f5b6885b4c-kube-api-access-2wdwz\") pod \"authorino-75d556bd77-v9ph6\" (UID: \"9a64b673-c726-44d1-af6c-65f5b6885b4c\") " pod="kuadrant-system/authorino-75d556bd77-v9ph6"
Jul 16 15:43:07.208216 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.208195    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbrdd\" (UniqueName: \"kubernetes.io/projected/f9f56ca9-643a-4da8-8608-1621ea57dced-kube-api-access-rbrdd\") pod \"authorino-58cd955675-rblf5\" (UID: \"f9f56ca9-643a-4da8-8608-1621ea57dced\") " pod="kuadrant-system/authorino-58cd955675-rblf5"
Jul 16 15:43:07.300519 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.300437    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wdwz\" (UniqueName: \"kubernetes.io/projected/9a64b673-c726-44d1-af6c-65f5b6885b4c-kube-api-access-2wdwz\") pod \"authorino-75d556bd77-v9ph6\" (UID: \"9a64b673-c726-44d1-af6c-65f5b6885b4c\") " pod="kuadrant-system/authorino-75d556bd77-v9ph6"
Jul 16 15:43:07.310781 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.310757    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wdwz\" (UniqueName: \"kubernetes.io/projected/9a64b673-c726-44d1-af6c-65f5b6885b4c-kube-api-access-2wdwz\") pod \"authorino-75d556bd77-v9ph6\" (UID: \"9a64b673-c726-44d1-af6c-65f5b6885b4c\") " pod="kuadrant-system/authorino-75d556bd77-v9ph6"
Jul 16 15:43:07.360551 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.360522    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rblf5"
Jul 16 15:43:07.421734 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.421706    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-v9ph6"
Jul 16 15:43:07.522293 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.522249    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rblf5"]
Jul 16 15:43:07.524963 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:43:07.524916    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9f56ca9_643a_4da8_8608_1621ea57dced.slice/crio-fb064ae1cb60549ec75aff5a1196773f4362d6e9707fed7e971d38d12a92279d WatchSource:0}: Error finding container fb064ae1cb60549ec75aff5a1196773f4362d6e9707fed7e971d38d12a92279d: Status 404 returned error can't find the container with id fb064ae1cb60549ec75aff5a1196773f4362d6e9707fed7e971d38d12a92279d
Jul 16 15:43:07.569262 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:07.569183    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v9ph6"]
Jul 16 15:43:07.571306 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:43:07.571268    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a64b673_c726_44d1_af6c_65f5b6885b4c.slice/crio-360df19f17b7c41d0ab0c5b0912bf538d1c7b876822c818a65475c4ff250f9b9 WatchSource:0}: Error finding container 360df19f17b7c41d0ab0c5b0912bf538d1c7b876822c818a65475c4ff250f9b9: Status 404 returned error can't find the container with id 360df19f17b7c41d0ab0c5b0912bf538d1c7b876822c818a65475c4ff250f9b9
Jul 16 15:43:08.128637 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:08.128342    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-v9ph6" event={"ID":"9a64b673-c726-44d1-af6c-65f5b6885b4c","Type":"ContainerStarted","Data":"360df19f17b7c41d0ab0c5b0912bf538d1c7b876822c818a65475c4ff250f9b9"}
Jul 16 15:43:08.129804 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:08.129775    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rblf5" event={"ID":"f9f56ca9-643a-4da8-8608-1621ea57dced","Type":"ContainerStarted","Data":"fb064ae1cb60549ec75aff5a1196773f4362d6e9707fed7e971d38d12a92279d"}
Jul 16 15:43:08.131147 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:08.131124    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n" event={"ID":"6e52fab9-13a6-4944-aaa9-38ec79c8873b","Type":"ContainerStarted","Data":"e1e21e47f18178d59fde034a0ff5639a1300e05a283db7eb5c7acb55e622393a"}
Jul 16 15:43:13.160168 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:13.160133    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-v9ph6" event={"ID":"9a64b673-c726-44d1-af6c-65f5b6885b4c","Type":"ContainerStarted","Data":"45dd3ebc929fb28fbde6d9d28eba773e71edc24bc9167ed472a2553a04e09690"}
Jul 16 15:43:13.161613 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:13.161590    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rblf5" event={"ID":"f9f56ca9-643a-4da8-8608-1621ea57dced","Type":"ContainerStarted","Data":"b3b735057ee6e026ee2f812bd8b2eb64eada1fb23278b73efc77ac6716a6f402"}
Jul 16 15:43:13.162870 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:13.162851    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n" event={"ID":"6e52fab9-13a6-4944-aaa9-38ec79c8873b","Type":"ContainerStarted","Data":"be9aa77c7a12186c434254827329c668ea9f8b327864bb7c453718a8c1ea45a2"}
Jul 16 15:43:13.162992 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:13.162973    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n"
Jul 16 15:43:13.175292 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:13.175253    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-v9ph6" podStartSLOduration=1.223547248 podStartE2EDuration="6.175241126s" totalImagesPullingTime="4.951693878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:43:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:43:13.174321471 +0000 UTC m=+459.363185425" watchObservedRunningTime="2026-07-16 15:43:13.175241126 +0000 UTC m=+459.364105058"
Jul 16 15:43:13.189600 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:13.189550    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-rblf5" podStartSLOduration=2.241845579 podStartE2EDuration="7.189535842s" totalImagesPullingTime="4.947690263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:43:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:43:13.188918671 +0000 UTC m=+459.377782627" watchObservedRunningTime="2026-07-16 15:43:13.189535842 +0000 UTC m=+459.378399799"
Jul 16 15:43:13.211326 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:13.211283    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n" podStartSLOduration=1.8673840130000001 podStartE2EDuration="7.211270269s" totalImagesPullingTime="5.343886256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:43:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:43:13.210542841 +0000 UTC m=+459.399406797" watchObservedRunningTime="2026-07-16 15:43:13.211270269 +0000 UTC m=+459.400134223"
Jul 16 15:43:13.218185 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:13.218156    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rblf5"]
Jul 16 15:43:15.171785 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:15.171743    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-rblf5" podUID="f9f56ca9-643a-4da8-8608-1621ea57dced" containerName="authorino" containerID="cri-o://b3b735057ee6e026ee2f812bd8b2eb64eada1fb23278b73efc77ac6716a6f402" gracePeriod=30
Jul 16 15:43:15.422870 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:15.422816    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rblf5"
Jul 16 15:43:15.474948 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:15.474906    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbrdd\" (UniqueName: \"kubernetes.io/projected/f9f56ca9-643a-4da8-8608-1621ea57dced-kube-api-access-rbrdd\") pod \"f9f56ca9-643a-4da8-8608-1621ea57dced\" (UID: \"f9f56ca9-643a-4da8-8608-1621ea57dced\") "
Jul 16 15:43:15.476811 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:15.476792    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9f56ca9-643a-4da8-8608-1621ea57dced-kube-api-access-rbrdd" (OuterVolumeSpecName: "kube-api-access-rbrdd") pod "f9f56ca9-643a-4da8-8608-1621ea57dced" (UID: "f9f56ca9-643a-4da8-8608-1621ea57dced"). InnerVolumeSpecName "kube-api-access-rbrdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:43:15.575479 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:15.575453    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbrdd\" (UniqueName: \"kubernetes.io/projected/f9f56ca9-643a-4da8-8608-1621ea57dced-kube-api-access-rbrdd\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:43:16.177942 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:16.177891    2570 generic.go:356] "Generic (PLEG): container finished" podID="f9f56ca9-643a-4da8-8608-1621ea57dced" containerID="b3b735057ee6e026ee2f812bd8b2eb64eada1fb23278b73efc77ac6716a6f402" exitCode=0
Jul 16 15:43:16.178358 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:16.177951    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rblf5"
Jul 16 15:43:16.178358 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:16.177966    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rblf5" event={"ID":"f9f56ca9-643a-4da8-8608-1621ea57dced","Type":"ContainerDied","Data":"b3b735057ee6e026ee2f812bd8b2eb64eada1fb23278b73efc77ac6716a6f402"}
Jul 16 15:43:16.178358 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:16.178002    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rblf5" event={"ID":"f9f56ca9-643a-4da8-8608-1621ea57dced","Type":"ContainerDied","Data":"fb064ae1cb60549ec75aff5a1196773f4362d6e9707fed7e971d38d12a92279d"}
Jul 16 15:43:16.178358 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:16.178025    2570 scope.go:117] "RemoveContainer" containerID="b3b735057ee6e026ee2f812bd8b2eb64eada1fb23278b73efc77ac6716a6f402"
Jul 16 15:43:16.189155 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:16.189117    2570 scope.go:117] "RemoveContainer" containerID="b3b735057ee6e026ee2f812bd8b2eb64eada1fb23278b73efc77ac6716a6f402"
Jul 16 15:43:16.189665 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:43:16.189639    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3b735057ee6e026ee2f812bd8b2eb64eada1fb23278b73efc77ac6716a6f402\": container with ID starting with b3b735057ee6e026ee2f812bd8b2eb64eada1fb23278b73efc77ac6716a6f402 not found: ID does not exist" containerID="b3b735057ee6e026ee2f812bd8b2eb64eada1fb23278b73efc77ac6716a6f402"
Jul 16 15:43:16.189743 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:16.189673    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3b735057ee6e026ee2f812bd8b2eb64eada1fb23278b73efc77ac6716a6f402"} err="failed to get container status \"b3b735057ee6e026ee2f812bd8b2eb64eada1fb23278b73efc77ac6716a6f402\": rpc error: code = NotFound desc = could not find container \"b3b735057ee6e026ee2f812bd8b2eb64eada1fb23278b73efc77ac6716a6f402\": container with ID starting with b3b735057ee6e026ee2f812bd8b2eb64eada1fb23278b73efc77ac6716a6f402 not found: ID does not exist"
Jul 16 15:43:16.206956 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:16.206916    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rblf5"]
Jul 16 15:43:16.210811 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:16.210792    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-rblf5"]
Jul 16 15:43:16.266736 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:16.266697    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9f56ca9-643a-4da8-8608-1621ea57dced" path="/var/lib/kubelet/pods/f9f56ca9-643a-4da8-8608-1621ea57dced/volumes"
Jul 16 15:43:21.238340 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:21.238293    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r245n"]
Jul 16 15:43:21.238774 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:21.238553    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n" podUID="6e52fab9-13a6-4944-aaa9-38ec79c8873b" containerName="limitador" containerID="cri-o://be9aa77c7a12186c434254827329c668ea9f8b327864bb7c453718a8c1ea45a2" gracePeriod=30
Jul 16 15:43:21.240686 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:21.240660    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n"
Jul 16 15:43:21.787269 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:21.787239    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n"
Jul 16 15:43:21.825024 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:21.824945    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6e52fab9-13a6-4944-aaa9-38ec79c8873b-config-file\") pod \"6e52fab9-13a6-4944-aaa9-38ec79c8873b\" (UID: \"6e52fab9-13a6-4944-aaa9-38ec79c8873b\") "
Jul 16 15:43:21.825024 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:21.825005    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpt7t\" (UniqueName: \"kubernetes.io/projected/6e52fab9-13a6-4944-aaa9-38ec79c8873b-kube-api-access-dpt7t\") pod \"6e52fab9-13a6-4944-aaa9-38ec79c8873b\" (UID: \"6e52fab9-13a6-4944-aaa9-38ec79c8873b\") "
Jul 16 15:43:21.825293 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:21.825271    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e52fab9-13a6-4944-aaa9-38ec79c8873b-config-file" (OuterVolumeSpecName: "config-file") pod "6e52fab9-13a6-4944-aaa9-38ec79c8873b" (UID: "6e52fab9-13a6-4944-aaa9-38ec79c8873b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 15:43:21.825394 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:21.825365    2570 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6e52fab9-13a6-4944-aaa9-38ec79c8873b-config-file\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:43:21.827108 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:21.827080    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e52fab9-13a6-4944-aaa9-38ec79c8873b-kube-api-access-dpt7t" (OuterVolumeSpecName: "kube-api-access-dpt7t") pod "6e52fab9-13a6-4944-aaa9-38ec79c8873b" (UID: "6e52fab9-13a6-4944-aaa9-38ec79c8873b"). InnerVolumeSpecName "kube-api-access-dpt7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:43:21.926659 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:21.926629    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpt7t\" (UniqueName: \"kubernetes.io/projected/6e52fab9-13a6-4944-aaa9-38ec79c8873b-kube-api-access-dpt7t\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:43:22.206534 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.206495    2570 generic.go:356] "Generic (PLEG): container finished" podID="6e52fab9-13a6-4944-aaa9-38ec79c8873b" containerID="be9aa77c7a12186c434254827329c668ea9f8b327864bb7c453718a8c1ea45a2" exitCode=0
Jul 16 15:43:22.206719 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.206545    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n"
Jul 16 15:43:22.206719 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.206577    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n" event={"ID":"6e52fab9-13a6-4944-aaa9-38ec79c8873b","Type":"ContainerDied","Data":"be9aa77c7a12186c434254827329c668ea9f8b327864bb7c453718a8c1ea45a2"}
Jul 16 15:43:22.206719 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.206633    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-r245n" event={"ID":"6e52fab9-13a6-4944-aaa9-38ec79c8873b","Type":"ContainerDied","Data":"e1e21e47f18178d59fde034a0ff5639a1300e05a283db7eb5c7acb55e622393a"}
Jul 16 15:43:22.206719 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.206657    2570 scope.go:117] "RemoveContainer" containerID="be9aa77c7a12186c434254827329c668ea9f8b327864bb7c453718a8c1ea45a2"
Jul 16 15:43:22.215785 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.215765    2570 scope.go:117] "RemoveContainer" containerID="be9aa77c7a12186c434254827329c668ea9f8b327864bb7c453718a8c1ea45a2"
Jul 16 15:43:22.216056 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:43:22.216036    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be9aa77c7a12186c434254827329c668ea9f8b327864bb7c453718a8c1ea45a2\": container with ID starting with be9aa77c7a12186c434254827329c668ea9f8b327864bb7c453718a8c1ea45a2 not found: ID does not exist" containerID="be9aa77c7a12186c434254827329c668ea9f8b327864bb7c453718a8c1ea45a2"
Jul 16 15:43:22.216107 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.216065    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be9aa77c7a12186c434254827329c668ea9f8b327864bb7c453718a8c1ea45a2"} err="failed to get container status \"be9aa77c7a12186c434254827329c668ea9f8b327864bb7c453718a8c1ea45a2\": rpc error: code = NotFound desc = could not find container \"be9aa77c7a12186c434254827329c668ea9f8b327864bb7c453718a8c1ea45a2\": container with ID starting with be9aa77c7a12186c434254827329c668ea9f8b327864bb7c453718a8c1ea45a2 not found: ID does not exist"
Jul 16 15:43:22.228526 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.228500    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r245n"]
Jul 16 15:43:22.232848 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.232826    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r245n"]
Jul 16 15:43:22.267469 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.267438    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e52fab9-13a6-4944-aaa9-38ec79c8873b" path="/var/lib/kubelet/pods/6e52fab9-13a6-4944-aaa9-38ec79c8873b/volumes"
Jul 16 15:43:22.311013 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.310981    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:22.410061 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.410028    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:22.738332 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.738298    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-swmwp"]
Jul 16 15:43:22.738702 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.738685    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e52fab9-13a6-4944-aaa9-38ec79c8873b" containerName="limitador"
Jul 16 15:43:22.738787 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.738705    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e52fab9-13a6-4944-aaa9-38ec79c8873b" containerName="limitador"
Jul 16 15:43:22.738787 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.738716    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9f56ca9-643a-4da8-8608-1621ea57dced" containerName="authorino"
Jul 16 15:43:22.738787 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.738724    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9f56ca9-643a-4da8-8608-1621ea57dced" containerName="authorino"
Jul 16 15:43:22.738951 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.738828    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9f56ca9-643a-4da8-8608-1621ea57dced" containerName="authorino"
Jul 16 15:43:22.738951 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.738839    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e52fab9-13a6-4944-aaa9-38ec79c8873b" containerName="limitador"
Jul 16 15:43:22.742898 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.742878    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-swmwp"
Jul 16 15:43:22.745670 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.745647    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 15:43:22.745670 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.745665    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 16 15:43:22.746775 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.746758    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-ffd2w\""
Jul 16 15:43:22.746863 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.746779    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 15:43:22.750175 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.750070    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-swmwp"]
Jul 16 15:43:22.834399 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.834353    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpc2f\" (UniqueName: \"kubernetes.io/projected/7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b-kube-api-access-rpc2f\") pod \"postgres-69fb4594cf-swmwp\" (UID: \"7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-swmwp"
Jul 16 15:43:22.834668 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.834528    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b-data\") pod \"postgres-69fb4594cf-swmwp\" (UID: \"7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-swmwp"
Jul 16 15:43:22.853221 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.853189    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:22.935442 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.935409    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpc2f\" (UniqueName: \"kubernetes.io/projected/7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b-kube-api-access-rpc2f\") pod \"postgres-69fb4594cf-swmwp\" (UID: \"7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-swmwp"
Jul 16 15:43:22.935624 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.935480    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b-data\") pod \"postgres-69fb4594cf-swmwp\" (UID: \"7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-swmwp"
Jul 16 15:43:22.935811 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.935795    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b-data\") pod \"postgres-69fb4594cf-swmwp\" (UID: \"7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-swmwp"
Jul 16 15:43:22.949096 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:22.949066    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpc2f\" (UniqueName: \"kubernetes.io/projected/7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b-kube-api-access-rpc2f\") pod \"postgres-69fb4594cf-swmwp\" (UID: \"7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-swmwp"
Jul 16 15:43:23.054184 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:23.054090    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-swmwp"
Jul 16 15:43:23.184985 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:23.184962    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-swmwp"]
Jul 16 15:43:23.187201 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:43:23.187175    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b36b36c_f9a0_4d6a_9311_c17ab19ffd7b.slice/crio-242b44883fdfdc652dbb80c8abae7910bad619d105bb0dbe3caaf7612274f43b WatchSource:0}: Error finding container 242b44883fdfdc652dbb80c8abae7910bad619d105bb0dbe3caaf7612274f43b: Status 404 returned error can't find the container with id 242b44883fdfdc652dbb80c8abae7910bad619d105bb0dbe3caaf7612274f43b
Jul 16 15:43:23.212378 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:23.212351    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-swmwp" event={"ID":"7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b","Type":"ContainerStarted","Data":"242b44883fdfdc652dbb80c8abae7910bad619d105bb0dbe3caaf7612274f43b"}
Jul 16 15:43:29.247984 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:29.247949    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-swmwp" event={"ID":"7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b","Type":"ContainerStarted","Data":"ab2a6a073ea471583a2cc65ea856d5f936223eba52e1e5d84da8fb8626b44298"}
Jul 16 15:43:29.248457 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:29.248029    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-swmwp"
Jul 16 15:43:29.266453 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:29.266393    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-swmwp" podStartSLOduration=2.09919426 podStartE2EDuration="7.266375406s" totalImagesPullingTime="5.167181146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:43:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:43:29.264984281 +0000 UTC m=+475.453848253" watchObservedRunningTime="2026-07-16 15:43:29.266375406 +0000 UTC m=+475.455239366"
Jul 16 15:43:31.723735 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:31.723693    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:31.825654 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:31.825625    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:35.281154 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:35.281125    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-swmwp"
Jul 16 15:43:35.602669 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:35.602640    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:35.874147 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:35.874063    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-szzlb"]
Jul 16 15:43:35.881951 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:35.881917    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-szzlb"
Jul 16 15:43:35.885806 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:35.885785    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-szzlb"]
Jul 16 15:43:35.943729 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:35.943689    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzpwh\" (UniqueName: \"kubernetes.io/projected/42e8a744-9696-4a6f-8cc3-6fdffa65cd42-kube-api-access-dzpwh\") pod \"authorino-5d4b67c446-szzlb\" (UID: \"42e8a744-9696-4a6f-8cc3-6fdffa65cd42\") " pod="kuadrant-system/authorino-5d4b67c446-szzlb"
Jul 16 15:43:36.044757 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.044720    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzpwh\" (UniqueName: \"kubernetes.io/projected/42e8a744-9696-4a6f-8cc3-6fdffa65cd42-kube-api-access-dzpwh\") pod \"authorino-5d4b67c446-szzlb\" (UID: \"42e8a744-9696-4a6f-8cc3-6fdffa65cd42\") " pod="kuadrant-system/authorino-5d4b67c446-szzlb"
Jul 16 15:43:36.055724 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.055691    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzpwh\" (UniqueName: \"kubernetes.io/projected/42e8a744-9696-4a6f-8cc3-6fdffa65cd42-kube-api-access-dzpwh\") pod \"authorino-5d4b67c446-szzlb\" (UID: \"42e8a744-9696-4a6f-8cc3-6fdffa65cd42\") " pod="kuadrant-system/authorino-5d4b67c446-szzlb"
Jul 16 15:43:36.098148 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.098108    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:36.159958 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.159862    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-szzlb"]
Jul 16 15:43:36.160174 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.160160    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-szzlb"
Jul 16 15:43:36.297892 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.293814    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-szzlb"]
Jul 16 15:43:36.300739 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:43:36.300711    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42e8a744_9696_4a6f_8cc3_6fdffa65cd42.slice/crio-8531c1858dfbcf973a94f56bb8f5af9ec664db26741708509fc61f36faafe3b3 WatchSource:0}: Error finding container 8531c1858dfbcf973a94f56bb8f5af9ec664db26741708509fc61f36faafe3b3: Status 404 returned error can't find the container with id 8531c1858dfbcf973a94f56bb8f5af9ec664db26741708509fc61f36faafe3b3
Jul 16 15:43:36.487402 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.487324    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7fbb79cdf7-6t5lw"]
Jul 16 15:43:36.492630 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.492611    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw"
Jul 16 15:43:36.494911 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.494889    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 16 15:43:36.498949 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.498913    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7fbb79cdf7-6t5lw"]
Jul 16 15:43:36.550051 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.550018    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cnt9\" (UniqueName: \"kubernetes.io/projected/daea8713-6f9f-4904-954c-b853ce89ee06-kube-api-access-7cnt9\") pod \"authorino-7fbb79cdf7-6t5lw\" (UID: \"daea8713-6f9f-4904-954c-b853ce89ee06\") " pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw"
Jul 16 15:43:36.550219 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.550083    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/daea8713-6f9f-4904-954c-b853ce89ee06-tls-cert\") pod \"authorino-7fbb79cdf7-6t5lw\" (UID: \"daea8713-6f9f-4904-954c-b853ce89ee06\") " pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw"
Jul 16 15:43:36.651511 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.651479    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cnt9\" (UniqueName: \"kubernetes.io/projected/daea8713-6f9f-4904-954c-b853ce89ee06-kube-api-access-7cnt9\") pod \"authorino-7fbb79cdf7-6t5lw\" (UID: \"daea8713-6f9f-4904-954c-b853ce89ee06\") " pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw"
Jul 16 15:43:36.651676 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.651542    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/daea8713-6f9f-4904-954c-b853ce89ee06-tls-cert\") pod \"authorino-7fbb79cdf7-6t5lw\" (UID: \"daea8713-6f9f-4904-954c-b853ce89ee06\") " pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw"
Jul 16 15:43:36.653885 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.653860    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/daea8713-6f9f-4904-954c-b853ce89ee06-tls-cert\") pod \"authorino-7fbb79cdf7-6t5lw\" (UID: \"daea8713-6f9f-4904-954c-b853ce89ee06\") " pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw"
Jul 16 15:43:36.662699 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.662676    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cnt9\" (UniqueName: \"kubernetes.io/projected/daea8713-6f9f-4904-954c-b853ce89ee06-kube-api-access-7cnt9\") pod \"authorino-7fbb79cdf7-6t5lw\" (UID: \"daea8713-6f9f-4904-954c-b853ce89ee06\") " pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw"
Jul 16 15:43:36.802919 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.802895    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw"
Jul 16 15:43:36.943700 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:36.943635    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7fbb79cdf7-6t5lw"]
Jul 16 15:43:36.946533 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:43:36.946503    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddaea8713_6f9f_4904_954c_b853ce89ee06.slice/crio-9f4810d369386921e3ba0401f3b31ff03082e0aadac18dd9189358d99ee82768 WatchSource:0}: Error finding container 9f4810d369386921e3ba0401f3b31ff03082e0aadac18dd9189358d99ee82768: Status 404 returned error can't find the container with id 9f4810d369386921e3ba0401f3b31ff03082e0aadac18dd9189358d99ee82768
Jul 16 15:43:37.285632 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:37.285587    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw" event={"ID":"daea8713-6f9f-4904-954c-b853ce89ee06","Type":"ContainerStarted","Data":"9f4810d369386921e3ba0401f3b31ff03082e0aadac18dd9189358d99ee82768"}
Jul 16 15:43:37.287010 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:37.286986    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-szzlb" event={"ID":"42e8a744-9696-4a6f-8cc3-6fdffa65cd42","Type":"ContainerStarted","Data":"274cee161224b321528e01f8db5759a56ce175bc2dc3377eecf6efaf4838e29c"}
Jul 16 15:43:37.287114 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:37.287023    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-szzlb" event={"ID":"42e8a744-9696-4a6f-8cc3-6fdffa65cd42","Type":"ContainerStarted","Data":"8531c1858dfbcf973a94f56bb8f5af9ec664db26741708509fc61f36faafe3b3"}
Jul 16 15:43:37.287114 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:37.287039    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-szzlb" podUID="42e8a744-9696-4a6f-8cc3-6fdffa65cd42" containerName="authorino" containerID="cri-o://274cee161224b321528e01f8db5759a56ce175bc2dc3377eecf6efaf4838e29c" gracePeriod=30
Jul 16 15:43:37.303122 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:37.303077    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-szzlb" podStartSLOduration=1.8633472009999998 podStartE2EDuration="2.303059453s" totalImagesPullingTime="439.712252ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:43:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:43:37.30187129 +0000 UTC m=+483.490735246" watchObservedRunningTime="2026-07-16 15:43:37.303059453 +0000 UTC m=+483.491923452"
Jul 16 15:43:37.606209 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:37.606179    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-szzlb"
Jul 16 15:43:37.761174 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:37.761139    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzpwh\" (UniqueName: \"kubernetes.io/projected/42e8a744-9696-4a6f-8cc3-6fdffa65cd42-kube-api-access-dzpwh\") pod \"42e8a744-9696-4a6f-8cc3-6fdffa65cd42\" (UID: \"42e8a744-9696-4a6f-8cc3-6fdffa65cd42\") "
Jul 16 15:43:37.763278 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:37.763250    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42e8a744-9696-4a6f-8cc3-6fdffa65cd42-kube-api-access-dzpwh" (OuterVolumeSpecName: "kube-api-access-dzpwh") pod "42e8a744-9696-4a6f-8cc3-6fdffa65cd42" (UID: "42e8a744-9696-4a6f-8cc3-6fdffa65cd42"). InnerVolumeSpecName "kube-api-access-dzpwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:43:37.862542 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:37.862512    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzpwh\" (UniqueName: \"kubernetes.io/projected/42e8a744-9696-4a6f-8cc3-6fdffa65cd42-kube-api-access-dzpwh\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:43:38.292103 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:38.292078    2570 generic.go:356] "Generic (PLEG): container finished" podID="42e8a744-9696-4a6f-8cc3-6fdffa65cd42" containerID="274cee161224b321528e01f8db5759a56ce175bc2dc3377eecf6efaf4838e29c" exitCode=0
Jul 16 15:43:38.292204 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:38.292120    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-szzlb"
Jul 16 15:43:38.292204 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:38.292162    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-szzlb" event={"ID":"42e8a744-9696-4a6f-8cc3-6fdffa65cd42","Type":"ContainerDied","Data":"274cee161224b321528e01f8db5759a56ce175bc2dc3377eecf6efaf4838e29c"}
Jul 16 15:43:38.292204 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:38.292199    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-szzlb" event={"ID":"42e8a744-9696-4a6f-8cc3-6fdffa65cd42","Type":"ContainerDied","Data":"8531c1858dfbcf973a94f56bb8f5af9ec664db26741708509fc61f36faafe3b3"}
Jul 16 15:43:38.292301 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:38.292215    2570 scope.go:117] "RemoveContainer" containerID="274cee161224b321528e01f8db5759a56ce175bc2dc3377eecf6efaf4838e29c"
Jul 16 15:43:38.302371 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:38.302352    2570 scope.go:117] "RemoveContainer" containerID="274cee161224b321528e01f8db5759a56ce175bc2dc3377eecf6efaf4838e29c"
Jul 16 15:43:38.302647 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:43:38.302629    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"274cee161224b321528e01f8db5759a56ce175bc2dc3377eecf6efaf4838e29c\": container with ID starting with 274cee161224b321528e01f8db5759a56ce175bc2dc3377eecf6efaf4838e29c not found: ID does not exist" containerID="274cee161224b321528e01f8db5759a56ce175bc2dc3377eecf6efaf4838e29c"
Jul 16 15:43:38.302714 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:38.302655    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"274cee161224b321528e01f8db5759a56ce175bc2dc3377eecf6efaf4838e29c"} err="failed to get container status \"274cee161224b321528e01f8db5759a56ce175bc2dc3377eecf6efaf4838e29c\": rpc error: code = NotFound desc = could not find container \"274cee161224b321528e01f8db5759a56ce175bc2dc3377eecf6efaf4838e29c\": container with ID starting with 274cee161224b321528e01f8db5759a56ce175bc2dc3377eecf6efaf4838e29c not found: ID does not exist"
Jul 16 15:43:38.308460 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:38.308439    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-szzlb"]
Jul 16 15:43:38.312484 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:38.312464    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-szzlb"]
Jul 16 15:43:39.298087 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:39.298054    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw" event={"ID":"daea8713-6f9f-4904-954c-b853ce89ee06","Type":"ContainerStarted","Data":"56d93d9c52c7e12932978b2cdcee561861bb52e527e759a6756bac28ca3fa241"}
Jul 16 15:43:39.314110 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:39.314062    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw" podStartSLOduration=1.99515404 podStartE2EDuration="3.314048171s" totalImagesPullingTime="1.318894131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:43:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:43:39.313626825 +0000 UTC m=+485.502490781" watchObservedRunningTime="2026-07-16 15:43:39.314048171 +0000 UTC m=+485.502912177"
Jul 16 15:43:39.341541 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:39.341503    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v9ph6"]
Jul 16 15:43:39.341750 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:39.341724    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-v9ph6" podUID="9a64b673-c726-44d1-af6c-65f5b6885b4c" containerName="authorino" containerID="cri-o://45dd3ebc929fb28fbde6d9d28eba773e71edc24bc9167ed472a2553a04e09690" gracePeriod=30
Jul 16 15:43:39.624572 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:39.624547    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-v9ph6"
Jul 16 15:43:39.777205 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:39.777175    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wdwz\" (UniqueName: \"kubernetes.io/projected/9a64b673-c726-44d1-af6c-65f5b6885b4c-kube-api-access-2wdwz\") pod \"9a64b673-c726-44d1-af6c-65f5b6885b4c\" (UID: \"9a64b673-c726-44d1-af6c-65f5b6885b4c\") "
Jul 16 15:43:39.779226 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:39.779197    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a64b673-c726-44d1-af6c-65f5b6885b4c-kube-api-access-2wdwz" (OuterVolumeSpecName: "kube-api-access-2wdwz") pod "9a64b673-c726-44d1-af6c-65f5b6885b4c" (UID: "9a64b673-c726-44d1-af6c-65f5b6885b4c"). InnerVolumeSpecName "kube-api-access-2wdwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:43:39.878697 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:39.878621    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wdwz\" (UniqueName: \"kubernetes.io/projected/9a64b673-c726-44d1-af6c-65f5b6885b4c-kube-api-access-2wdwz\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:43:40.267144 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:40.267063    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42e8a744-9696-4a6f-8cc3-6fdffa65cd42" path="/var/lib/kubelet/pods/42e8a744-9696-4a6f-8cc3-6fdffa65cd42/volumes"
Jul 16 15:43:40.304388 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:40.304358    2570 generic.go:356] "Generic (PLEG): container finished" podID="9a64b673-c726-44d1-af6c-65f5b6885b4c" containerID="45dd3ebc929fb28fbde6d9d28eba773e71edc24bc9167ed472a2553a04e09690" exitCode=0
Jul 16 15:43:40.304587 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:40.304404    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-v9ph6"
Jul 16 15:43:40.304587 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:40.304449    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-v9ph6" event={"ID":"9a64b673-c726-44d1-af6c-65f5b6885b4c","Type":"ContainerDied","Data":"45dd3ebc929fb28fbde6d9d28eba773e71edc24bc9167ed472a2553a04e09690"}
Jul 16 15:43:40.304587 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:40.304492    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-v9ph6" event={"ID":"9a64b673-c726-44d1-af6c-65f5b6885b4c","Type":"ContainerDied","Data":"360df19f17b7c41d0ab0c5b0912bf538d1c7b876822c818a65475c4ff250f9b9"}
Jul 16 15:43:40.304587 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:40.304514    2570 scope.go:117] "RemoveContainer" containerID="45dd3ebc929fb28fbde6d9d28eba773e71edc24bc9167ed472a2553a04e09690"
Jul 16 15:43:40.312441 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:40.312424    2570 scope.go:117] "RemoveContainer" containerID="45dd3ebc929fb28fbde6d9d28eba773e71edc24bc9167ed472a2553a04e09690"
Jul 16 15:43:40.312700 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:43:40.312680    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45dd3ebc929fb28fbde6d9d28eba773e71edc24bc9167ed472a2553a04e09690\": container with ID starting with 45dd3ebc929fb28fbde6d9d28eba773e71edc24bc9167ed472a2553a04e09690 not found: ID does not exist" containerID="45dd3ebc929fb28fbde6d9d28eba773e71edc24bc9167ed472a2553a04e09690"
Jul 16 15:43:40.312743 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:40.312708    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45dd3ebc929fb28fbde6d9d28eba773e71edc24bc9167ed472a2553a04e09690"} err="failed to get container status \"45dd3ebc929fb28fbde6d9d28eba773e71edc24bc9167ed472a2553a04e09690\": rpc error: code = NotFound desc = could not find container \"45dd3ebc929fb28fbde6d9d28eba773e71edc24bc9167ed472a2553a04e09690\": container with ID starting with 45dd3ebc929fb28fbde6d9d28eba773e71edc24bc9167ed472a2553a04e09690 not found: ID does not exist"
Jul 16 15:43:40.321662 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:40.321611    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v9ph6"]
Jul 16 15:43:40.323125 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:40.323107    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v9ph6"]
Jul 16 15:43:42.266488 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:42.266454    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a64b673-c726-44d1-af6c-65f5b6885b4c" path="/var/lib/kubelet/pods/9a64b673-c726-44d1-af6c-65f5b6885b4c/volumes"
Jul 16 15:43:43.960456 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:43.960425    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:44.059289 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.059243    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:44.149988 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.149950    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:44.150892 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.150872    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-775f5d4dfb-kzklc"]
Jul 16 15:43:44.151218 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.151205    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a64b673-c726-44d1-af6c-65f5b6885b4c" containerName="authorino"
Jul 16 15:43:44.151218 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.151219    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a64b673-c726-44d1-af6c-65f5b6885b4c" containerName="authorino"
Jul 16 15:43:44.151330 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.151231    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42e8a744-9696-4a6f-8cc3-6fdffa65cd42" containerName="authorino"
Jul 16 15:43:44.151330 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.151236    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e8a744-9696-4a6f-8cc3-6fdffa65cd42" containerName="authorino"
Jul 16 15:43:44.151330 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.151297    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="42e8a744-9696-4a6f-8cc3-6fdffa65cd42" containerName="authorino"
Jul 16 15:43:44.151330 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.151307    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a64b673-c726-44d1-af6c-65f5b6885b4c" containerName="authorino"
Jul 16 15:43:44.165175 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.165150    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-775f5d4dfb-kzklc"]
Jul 16 15:43:44.165353 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.165244    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-775f5d4dfb-kzklc"
Jul 16 15:43:44.168182 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.168161    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lf24x\""
Jul 16 15:43:44.168298 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.168204    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 15:43:44.168298 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.168219    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 15:43:44.243575 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.243483    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:44.315016 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.314986    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/927f8abf-4fa8-4d94-8e0a-52b61b3c154f-webhook-cert\") pod \"maas-controller-775f5d4dfb-kzklc\" (UID: \"927f8abf-4fa8-4d94-8e0a-52b61b3c154f\") " pod="opendatahub/maas-controller-775f5d4dfb-kzklc"
Jul 16 15:43:44.315183 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.315022    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zh9z\" (UniqueName: \"kubernetes.io/projected/927f8abf-4fa8-4d94-8e0a-52b61b3c154f-kube-api-access-6zh9z\") pod \"maas-controller-775f5d4dfb-kzklc\" (UID: \"927f8abf-4fa8-4d94-8e0a-52b61b3c154f\") " pod="opendatahub/maas-controller-775f5d4dfb-kzklc"
Jul 16 15:43:44.415917 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.415882    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zh9z\" (UniqueName: \"kubernetes.io/projected/927f8abf-4fa8-4d94-8e0a-52b61b3c154f-kube-api-access-6zh9z\") pod \"maas-controller-775f5d4dfb-kzklc\" (UID: \"927f8abf-4fa8-4d94-8e0a-52b61b3c154f\") " pod="opendatahub/maas-controller-775f5d4dfb-kzklc"
Jul 16 15:43:44.416091 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.416006    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/927f8abf-4fa8-4d94-8e0a-52b61b3c154f-webhook-cert\") pod \"maas-controller-775f5d4dfb-kzklc\" (UID: \"927f8abf-4fa8-4d94-8e0a-52b61b3c154f\") " pod="opendatahub/maas-controller-775f5d4dfb-kzklc"
Jul 16 15:43:44.418325 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.418301    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/927f8abf-4fa8-4d94-8e0a-52b61b3c154f-webhook-cert\") pod \"maas-controller-775f5d4dfb-kzklc\" (UID: \"927f8abf-4fa8-4d94-8e0a-52b61b3c154f\") " pod="opendatahub/maas-controller-775f5d4dfb-kzklc"
Jul 16 15:43:44.424678 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.424651    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zh9z\" (UniqueName: \"kubernetes.io/projected/927f8abf-4fa8-4d94-8e0a-52b61b3c154f-kube-api-access-6zh9z\") pod \"maas-controller-775f5d4dfb-kzklc\" (UID: \"927f8abf-4fa8-4d94-8e0a-52b61b3c154f\") " pod="opendatahub/maas-controller-775f5d4dfb-kzklc"
Jul 16 15:43:44.478675 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.478633    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-775f5d4dfb-kzklc"
Jul 16 15:43:44.610634 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:44.610573    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-775f5d4dfb-kzklc"]
Jul 16 15:43:44.612954 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:43:44.612897    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod927f8abf_4fa8_4d94_8e0a_52b61b3c154f.slice/crio-ce832000a230ca084d83b3a83f90bd730b389f9a71d5dada6517860a0819b51f WatchSource:0}: Error finding container ce832000a230ca084d83b3a83f90bd730b389f9a71d5dada6517860a0819b51f: Status 404 returned error can't find the container with id ce832000a230ca084d83b3a83f90bd730b389f9a71d5dada6517860a0819b51f
Jul 16 15:43:45.327797 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:45.327768    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-775f5d4dfb-kzklc" event={"ID":"927f8abf-4fa8-4d94-8e0a-52b61b3c154f","Type":"ContainerStarted","Data":"ce832000a230ca084d83b3a83f90bd730b389f9a71d5dada6517860a0819b51f"}
Jul 16 15:43:47.644953 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:47.644849    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:47.725008 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:47.724971    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:43:49.350158 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:49.350117    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-775f5d4dfb-kzklc" event={"ID":"927f8abf-4fa8-4d94-8e0a-52b61b3c154f","Type":"ContainerStarted","Data":"5d489251196e6a9c9e061f20b10aa055f5c462b38cd3e0f4d2762d71f93137d6"}
Jul 16 15:43:49.350629 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:49.350239    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-775f5d4dfb-kzklc"
Jul 16 15:43:49.368698 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:43:49.368645    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-775f5d4dfb-kzklc" podStartSLOduration=1.381406755 podStartE2EDuration="5.368628568s" totalImagesPullingTime="3.987221813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:43:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:43:49.367903517 +0000 UTC m=+495.556767476" watchObservedRunningTime="2026-07-16 15:43:49.368628568 +0000 UTC m=+495.557492524"
Jul 16 15:44:00.360402 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:00.360368    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-775f5d4dfb-kzklc"
Jul 16 15:44:19.935870 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:19.935822    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:44:20.435252 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:20.435199    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:44:20.625004 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:20.624974    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:44:21.004050 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.004008    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:44:21.092007 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.091972    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-kpmn2"]
Jul 16 15:44:21.095972 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.095945    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:44:21.098194 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.098175    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-kpmn2"
Jul 16 15:44:21.100758 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.100740    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 16 15:44:21.104815 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.104787    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-kpmn2"]
Jul 16 15:44:21.141137 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.141106    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-84dwr"]
Jul 16 15:44:21.144279 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.144262    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-84dwr"
Jul 16 15:44:21.147005 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.146985    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7z28k\""
Jul 16 15:44:21.154470 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.154449    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-84dwr"]
Jul 16 15:44:21.222473 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.222439    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a82ac839-3e06-4cae-b183-f2ac59856423-plugins-config-volume\") pod \"payload-pre-processing-7964457696-kpmn2\" (UID: \"a82ac839-3e06-4cae-b183-f2ac59856423\") " pod="openshift-ingress/payload-pre-processing-7964457696-kpmn2"
Jul 16 15:44:21.222659 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.222504    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2nnq\" (UniqueName: \"kubernetes.io/projected/1c89cea7-abe8-489e-86ef-0d979a20d8db-kube-api-access-t2nnq\") pod \"payload-processing-58f86c7798-84dwr\" (UID: \"1c89cea7-abe8-489e-86ef-0d979a20d8db\") " pod="openshift-ingress/payload-processing-58f86c7798-84dwr"
Jul 16 15:44:21.222659 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.222559    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q62s\" (UniqueName: \"kubernetes.io/projected/a82ac839-3e06-4cae-b183-f2ac59856423-kube-api-access-8q62s\") pod \"payload-pre-processing-7964457696-kpmn2\" (UID: \"a82ac839-3e06-4cae-b183-f2ac59856423\") " pod="openshift-ingress/payload-pre-processing-7964457696-kpmn2"
Jul 16 15:44:21.222659 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.222626    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c89cea7-abe8-489e-86ef-0d979a20d8db-plugins-config-volume\") pod \"payload-processing-58f86c7798-84dwr\" (UID: \"1c89cea7-abe8-489e-86ef-0d979a20d8db\") " pod="openshift-ingress/payload-processing-58f86c7798-84dwr"
Jul 16 15:44:21.323916 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.323836    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q62s\" (UniqueName: \"kubernetes.io/projected/a82ac839-3e06-4cae-b183-f2ac59856423-kube-api-access-8q62s\") pod \"payload-pre-processing-7964457696-kpmn2\" (UID: \"a82ac839-3e06-4cae-b183-f2ac59856423\") " pod="openshift-ingress/payload-pre-processing-7964457696-kpmn2"
Jul 16 15:44:21.323916 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.323876    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c89cea7-abe8-489e-86ef-0d979a20d8db-plugins-config-volume\") pod \"payload-processing-58f86c7798-84dwr\" (UID: \"1c89cea7-abe8-489e-86ef-0d979a20d8db\") " pod="openshift-ingress/payload-processing-58f86c7798-84dwr"
Jul 16 15:44:21.324143 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.323962    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a82ac839-3e06-4cae-b183-f2ac59856423-plugins-config-volume\") pod \"payload-pre-processing-7964457696-kpmn2\" (UID: \"a82ac839-3e06-4cae-b183-f2ac59856423\") " pod="openshift-ingress/payload-pre-processing-7964457696-kpmn2"
Jul 16 15:44:21.324143 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.324010    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2nnq\" (UniqueName: \"kubernetes.io/projected/1c89cea7-abe8-489e-86ef-0d979a20d8db-kube-api-access-t2nnq\") pod \"payload-processing-58f86c7798-84dwr\" (UID: \"1c89cea7-abe8-489e-86ef-0d979a20d8db\") " pod="openshift-ingress/payload-processing-58f86c7798-84dwr"
Jul 16 15:44:21.324590 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.324565    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c89cea7-abe8-489e-86ef-0d979a20d8db-plugins-config-volume\") pod \"payload-processing-58f86c7798-84dwr\" (UID: \"1c89cea7-abe8-489e-86ef-0d979a20d8db\") " pod="openshift-ingress/payload-processing-58f86c7798-84dwr"
Jul 16 15:44:21.324754 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.324734    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a82ac839-3e06-4cae-b183-f2ac59856423-plugins-config-volume\") pod \"payload-pre-processing-7964457696-kpmn2\" (UID: \"a82ac839-3e06-4cae-b183-f2ac59856423\") " pod="openshift-ingress/payload-pre-processing-7964457696-kpmn2"
Jul 16 15:44:21.338254 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.338227    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q62s\" (UniqueName: \"kubernetes.io/projected/a82ac839-3e06-4cae-b183-f2ac59856423-kube-api-access-8q62s\") pod \"payload-pre-processing-7964457696-kpmn2\" (UID: \"a82ac839-3e06-4cae-b183-f2ac59856423\") " pod="openshift-ingress/payload-pre-processing-7964457696-kpmn2"
Jul 16 15:44:21.339610 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.339590    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2nnq\" (UniqueName: \"kubernetes.io/projected/1c89cea7-abe8-489e-86ef-0d979a20d8db-kube-api-access-t2nnq\") pod \"payload-processing-58f86c7798-84dwr\" (UID: \"1c89cea7-abe8-489e-86ef-0d979a20d8db\") " pod="openshift-ingress/payload-processing-58f86c7798-84dwr"
Jul 16 15:44:21.409998 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.409969    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-kpmn2"
Jul 16 15:44:21.454408 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.454375    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-84dwr"
Jul 16 15:44:21.552675 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.552621    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-kpmn2"]
Jul 16 15:44:21.554547 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:44:21.554516    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda82ac839_3e06_4cae_b183_f2ac59856423.slice/crio-8802b2edbd35667be2ae86aa0401acdf9f6fce8ea6ef96ce707c70722d166069 WatchSource:0}: Error finding container 8802b2edbd35667be2ae86aa0401acdf9f6fce8ea6ef96ce707c70722d166069: Status 404 returned error can't find the container with id 8802b2edbd35667be2ae86aa0401acdf9f6fce8ea6ef96ce707c70722d166069
Jul 16 15:44:21.599090 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:21.599062    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-84dwr"]
Jul 16 15:44:21.601391 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:44:21.601366    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c89cea7_abe8_489e_86ef_0d979a20d8db.slice/crio-ea7c0c63f32f468525875b5704a9e57fd365d686e426ff3cfa9f86fafe651774 WatchSource:0}: Error finding container ea7c0c63f32f468525875b5704a9e57fd365d686e426ff3cfa9f86fafe651774: Status 404 returned error can't find the container with id ea7c0c63f32f468525875b5704a9e57fd365d686e426ff3cfa9f86fafe651774
Jul 16 15:44:22.503020 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:22.502979    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-kpmn2" event={"ID":"a82ac839-3e06-4cae-b183-f2ac59856423","Type":"ContainerStarted","Data":"8802b2edbd35667be2ae86aa0401acdf9f6fce8ea6ef96ce707c70722d166069"}
Jul 16 15:44:22.504749 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:22.504679    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-84dwr" event={"ID":"1c89cea7-abe8-489e-86ef-0d979a20d8db","Type":"ContainerStarted","Data":"ea7c0c63f32f468525875b5704a9e57fd365d686e426ff3cfa9f86fafe651774"}
Jul 16 15:44:23.511197 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:23.511162    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-kpmn2" event={"ID":"a82ac839-3e06-4cae-b183-f2ac59856423","Type":"ContainerStarted","Data":"8adc356089db27e4dd3525f4ee09b32b2efe9b2aaf1f9150ea27fed03f7ae4d7"}
Jul 16 15:44:23.511608 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:23.511248    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-kpmn2"
Jul 16 15:44:23.512532 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:23.512511    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-84dwr" event={"ID":"1c89cea7-abe8-489e-86ef-0d979a20d8db","Type":"ContainerStarted","Data":"c4f0126b456ed7d74f3c90935260b590301fedf6e57d6a61f61f8f752f2067e7"}
Jul 16 15:44:23.512625 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:23.512549    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-84dwr"
Jul 16 15:44:23.530366 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:23.530291    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-kpmn2" podStartSLOduration=0.714478298 podStartE2EDuration="2.530273383s" totalImagesPullingTime="1.815795085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:44:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:44:23.527601879 +0000 UTC m=+529.716465872" watchObservedRunningTime="2026-07-16 15:44:23.530273383 +0000 UTC m=+529.719137339"
Jul 16 15:44:29.549984 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:29.549952    2570 generic.go:356] "Generic (PLEG): container finished" podID="ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c" containerID="8fbd2348e2ce9d5d0cbdc687850db16c3a03b51906363cdb88e0ab72805c9cc8" exitCode=0
Jul 16 15:44:29.550414 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:29.550018    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl" event={"ID":"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c","Type":"ContainerDied","Data":"8fbd2348e2ce9d5d0cbdc687850db16c3a03b51906363cdb88e0ab72805c9cc8"}
Jul 16 15:44:29.550414 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:29.550389    2570 scope.go:117] "RemoveContainer" containerID="8fbd2348e2ce9d5d0cbdc687850db16c3a03b51906363cdb88e0ab72805c9cc8"
Jul 16 15:44:29.551357 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:29.551332    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 15:44:29.551459 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:29.551397    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 15:44:29.551459 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:29.551427    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 15:44:29.572349 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:29.572303    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-84dwr" podStartSLOduration=6.805800075 podStartE2EDuration="8.572287252s" totalImagesPullingTime="1.766487177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:44:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:44:23.550004777 +0000 UTC m=+529.738868730" watchObservedRunningTime="2026-07-16 15:44:29.572287252 +0000 UTC m=+535.761151206"
Jul 16 15:44:29.889528 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:29.889491    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:44:29.995918 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:29.995883    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:44:30.557743 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:30.557707    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl" event={"ID":"ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c","Type":"ContainerStarted","Data":"b30b790c8c81be540977ace9b82d9a2203755a9efaac77cd20b81adc952fb335"}
Jul 16 15:44:31.231951 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:31.231899    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:44:31.232153 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:31.232052    2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rcsbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 16 15:44:31.232153 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:31.232099    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl" podUID="ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 16 15:44:32.231470 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:32.231424    2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rcsbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 16 15:44:32.232007 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:32.231510    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl" podUID="ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 16 15:44:33.231447 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:33.231403    2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rcsbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 16 15:44:33.231672 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:33.231477    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl" podUID="ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 16 15:44:34.238306 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:34.238272    2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rcsbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 16 15:44:34.238723 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:34.238350    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl" podUID="ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 16 15:44:34.528868 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:34.528776    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-84dwr"
Jul 16 15:44:34.529470 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:34.529442    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-kpmn2"
Jul 16 15:44:35.235034 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:35.235004    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:44:35.235319 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:35.235292    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:44:35.236024 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:35.236005    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rcsbl"
Jul 16 15:44:36.700166 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.700134    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"]
Jul 16 15:44:36.705339 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.705319    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.709142 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.709121    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p8txr\""
Jul 16 15:44:36.709303 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.709123    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 16 15:44:36.709434 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.709123    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 16 15:44:36.709434 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.709185    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 16 15:44:36.713782 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.713762    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"]
Jul 16 15:44:36.729324 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.729297    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:44:36.781654 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.781628    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.781750 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.781662    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.781750 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.781709    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.781750 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.781743    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6tl6\" (UniqueName: \"kubernetes.io/projected/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-kube-api-access-r6tl6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.781843 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.781782    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.781843 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.781798    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.882881 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.882842    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.882881 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.882884    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6tl6\" (UniqueName: \"kubernetes.io/projected/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-kube-api-access-r6tl6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.883143 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.882963    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.883143 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.882986    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.883143 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.883042    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.883143 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.883070    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.883399 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.883374    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.883466 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.883401    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.883466 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.883426    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.885158 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.885133    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.885306 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.885290    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:36.891230 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:36.891209    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6tl6\" (UniqueName: \"kubernetes.io/projected/6322f4a6-f0f2-494c-a83e-b59ecb5eac30-kube-api-access-r6tl6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk\" (UID: \"6322f4a6-f0f2-494c-a83e-b59ecb5eac30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:37.017520 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:37.017447    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:37.156188 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:37.156167    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"]
Jul 16 15:44:37.158781 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:44:37.158754    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6322f4a6_f0f2_494c_a83e_b59ecb5eac30.slice/crio-ac8550d1b930eda75341319d369c8931852a613d678ddca595776d285e6a6a4e WatchSource:0}: Error finding container ac8550d1b930eda75341319d369c8931852a613d678ddca595776d285e6a6a4e: Status 404 returned error can't find the container with id ac8550d1b930eda75341319d369c8931852a613d678ddca595776d285e6a6a4e
Jul 16 15:44:37.594702 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:37.594628    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk" event={"ID":"6322f4a6-f0f2-494c-a83e-b59ecb5eac30","Type":"ContainerStarted","Data":"ac8550d1b930eda75341319d369c8931852a613d678ddca595776d285e6a6a4e"}
Jul 16 15:44:38.599695 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:38.599654    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk" event={"ID":"6322f4a6-f0f2-494c-a83e-b59ecb5eac30","Type":"ContainerStarted","Data":"bb4621f8344d04a30876f0b89ecad31405b9041e853c64aae5bfa4cd2592e965"}
Jul 16 15:44:38.620142 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:38.620101    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk" podStartSLOduration=1.4717085810000001 podStartE2EDuration="2.62008465s" totalImagesPullingTime="1.148376069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:44:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:44:38.617481721 +0000 UTC m=+544.806345675" watchObservedRunningTime="2026-07-16 15:44:38.62008465 +0000 UTC m=+544.808948604"
Jul 16 15:44:43.024551 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:43.024491    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:44:47.018232 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:47.018202    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:47.018667 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:47.018484    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:47.030190 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:47.030168    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:47.651585 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:47.651548    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk"
Jul 16 15:44:52.867065 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:52.867037    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:44:59.109868 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.109834    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"]
Jul 16 15:44:59.113254 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.113231    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.115662 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.115641    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 16 15:44:59.123203 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.123179    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"]
Jul 16 15:44:59.144390 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.144362    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:44:59.181173 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.181145    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67545190-9949-4ed0-a636-2f34f144d73c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.181277 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.181180    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbqvc\" (UniqueName: \"kubernetes.io/projected/67545190-9949-4ed0-a636-2f34f144d73c-kube-api-access-gbqvc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.181332 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.181268    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67545190-9949-4ed0-a636-2f34f144d73c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.181372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.181328    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67545190-9949-4ed0-a636-2f34f144d73c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.181372 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.181358    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67545190-9949-4ed0-a636-2f34f144d73c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.181451 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.181374    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67545190-9949-4ed0-a636-2f34f144d73c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.282299 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.282255    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67545190-9949-4ed0-a636-2f34f144d73c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.282299 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.282299    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbqvc\" (UniqueName: \"kubernetes.io/projected/67545190-9949-4ed0-a636-2f34f144d73c-kube-api-access-gbqvc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.282572 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.282378    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67545190-9949-4ed0-a636-2f34f144d73c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.282572 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.282402    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67545190-9949-4ed0-a636-2f34f144d73c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.282572 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.282433    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67545190-9949-4ed0-a636-2f34f144d73c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.282572 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.282455    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67545190-9949-4ed0-a636-2f34f144d73c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.282791 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.282764    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67545190-9949-4ed0-a636-2f34f144d73c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.282980 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.282827    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67545190-9949-4ed0-a636-2f34f144d73c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.282980 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.282969    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67545190-9949-4ed0-a636-2f34f144d73c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.285200 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.285177    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67545190-9949-4ed0-a636-2f34f144d73c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.285333 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.285316    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67545190-9949-4ed0-a636-2f34f144d73c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.292145 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.292125    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbqvc\" (UniqueName: \"kubernetes.io/projected/67545190-9949-4ed0-a636-2f34f144d73c-kube-api-access-gbqvc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzs5h\" (UID: \"67545190-9949-4ed0-a636-2f34f144d73c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.424456 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.424373    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:44:59.555505 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.555479    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"]
Jul 16 15:44:59.557364 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:44:59.557329    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67545190_9949_4ed0_a636_2f34f144d73c.slice/crio-215c81efb05e023c59c03f89fa372fa757164e3fa7c3788d7f415ef3d165356f WatchSource:0}: Error finding container 215c81efb05e023c59c03f89fa372fa757164e3fa7c3788d7f415ef3d165356f: Status 404 returned error can't find the container with id 215c81efb05e023c59c03f89fa372fa757164e3fa7c3788d7f415ef3d165356f
Jul 16 15:44:59.701725 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:44:59.701615    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h" event={"ID":"67545190-9949-4ed0-a636-2f34f144d73c","Type":"ContainerStarted","Data":"215c81efb05e023c59c03f89fa372fa757164e3fa7c3788d7f415ef3d165356f"}
Jul 16 15:45:00.710835 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:00.710799    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h" event={"ID":"67545190-9949-4ed0-a636-2f34f144d73c","Type":"ContainerStarted","Data":"978a18ed2dfa86e605709d2073d9ccebdd9e2d8c4770855e37303d0a6ed275e1"}
Jul 16 15:45:00.736676 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:00.736619    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h" podStartSLOduration=1.530048355 podStartE2EDuration="1.736601825s" totalImagesPullingTime="206.55347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:44:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:45:00.733606509 +0000 UTC m=+566.922470463" watchObservedRunningTime="2026-07-16 15:45:00.736601825 +0000 UTC m=+566.925465781"
Jul 16 15:45:06.816813 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:06.816772    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:45:09.425114 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:09.425082    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:45:09.425603 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:09.425156    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:45:09.437196 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:09.437174    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:45:09.761089 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:09.761004    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzs5h"
Jul 16 15:45:18.246696 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:18.246660    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:45:22.041670 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:22.041629    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:45:22.092393 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:22.092359    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:45:26.309504 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.309468    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"]
Jul 16 15:45:26.314692 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.314667    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.318204 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.318182    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 16 15:45:26.324285 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.324261    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"]
Jul 16 15:45:26.346619 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.346580    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:45:26.431096 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.431064    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4689b43c-42fd-418d-866b-945e55bb5d66-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.431096 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.431097    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fntw\" (UniqueName: \"kubernetes.io/projected/4689b43c-42fd-418d-866b-945e55bb5d66-kube-api-access-7fntw\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.431347 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.431129    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4689b43c-42fd-418d-866b-945e55bb5d66-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.431347 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.431146    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4689b43c-42fd-418d-866b-945e55bb5d66-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.431347 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.431241    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4689b43c-42fd-418d-866b-945e55bb5d66-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.431347 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.431297    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4689b43c-42fd-418d-866b-945e55bb5d66-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.531746 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.531717    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4689b43c-42fd-418d-866b-945e55bb5d66-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.531965 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.531753    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4689b43c-42fd-418d-866b-945e55bb5d66-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.531965 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.531784    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4689b43c-42fd-418d-866b-945e55bb5d66-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.531965 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.531813    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4689b43c-42fd-418d-866b-945e55bb5d66-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.531965 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.531860    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4689b43c-42fd-418d-866b-945e55bb5d66-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.531965 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.531877    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fntw\" (UniqueName: \"kubernetes.io/projected/4689b43c-42fd-418d-866b-945e55bb5d66-kube-api-access-7fntw\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.532254 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.532099    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4689b43c-42fd-418d-866b-945e55bb5d66-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.532254 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.532191    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4689b43c-42fd-418d-866b-945e55bb5d66-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.532390 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.532301    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4689b43c-42fd-418d-866b-945e55bb5d66-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.534079 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.534059    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4689b43c-42fd-418d-866b-945e55bb5d66-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.534246 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.534230    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4689b43c-42fd-418d-866b-945e55bb5d66-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.540173 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.540151    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fntw\" (UniqueName: \"kubernetes.io/projected/4689b43c-42fd-418d-866b-945e55bb5d66-kube-api-access-7fntw\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4\" (UID: \"4689b43c-42fd-418d-866b-945e55bb5d66\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.625396 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.625362    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:26.757889 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.757866    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"]
Jul 16 15:45:26.760111 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:45:26.760077    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4689b43c_42fd_418d_866b_945e55bb5d66.slice/crio-7c554a8ad5e5afa7d52dc7636b023a0eb35552ca8fd2b2b348788a3768b61920 WatchSource:0}: Error finding container 7c554a8ad5e5afa7d52dc7636b023a0eb35552ca8fd2b2b348788a3768b61920: Status 404 returned error can't find the container with id 7c554a8ad5e5afa7d52dc7636b023a0eb35552ca8fd2b2b348788a3768b61920
Jul 16 15:45:26.834315 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:26.834279    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4" event={"ID":"4689b43c-42fd-418d-866b-945e55bb5d66","Type":"ContainerStarted","Data":"7c554a8ad5e5afa7d52dc7636b023a0eb35552ca8fd2b2b348788a3768b61920"}
Jul 16 15:45:27.839715 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:27.839684    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4" event={"ID":"4689b43c-42fd-418d-866b-945e55bb5d66","Type":"ContainerStarted","Data":"cfc1c5e75590a0bc2a0e86b1f71b3d3c868cb717101209b115d2599626e384f2"}
Jul 16 15:45:27.859215 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:27.859164    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4" podStartSLOduration=1.634803239 podStartE2EDuration="1.859148661s" totalImagesPullingTime="224.345422ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:45:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:45:27.856706991 +0000 UTC m=+594.045570959" watchObservedRunningTime="2026-07-16 15:45:27.859148661 +0000 UTC m=+594.048012615"
Jul 16 15:45:29.915432 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:29.915393    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:45:34.173355 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:34.173329    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 15:45:34.173956 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:34.173650    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 15:45:36.626253 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:36.626208    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:36.626654 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:36.626279    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:36.639071 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:36.639042    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:36.892664 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:36.892584    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4"
Jul 16 15:45:41.520555 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:45:41.520514    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:46:09.701279 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:09.701159    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:46:09.797421 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:09.797390    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:46:10.432521 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:10.432488    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7fbb79cdf7-6t5lw"]
Jul 16 15:46:10.967759 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:10.967723    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:46:11.048760 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:11.048686    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw" podUID="daea8713-6f9f-4904-954c-b853ce89ee06" containerName="authorino" containerID="cri-o://56d93d9c52c7e12932978b2cdcee561861bb52e527e759a6756bac28ca3fa241" gracePeriod=30
Jul 16 15:46:11.068298 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:11.068258    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:46:11.305953 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:11.305917    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw"
Jul 16 15:46:11.445827 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:11.445794    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/daea8713-6f9f-4904-954c-b853ce89ee06-tls-cert\") pod \"daea8713-6f9f-4904-954c-b853ce89ee06\" (UID: \"daea8713-6f9f-4904-954c-b853ce89ee06\") "
Jul 16 15:46:11.446034 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:11.445839    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cnt9\" (UniqueName: \"kubernetes.io/projected/daea8713-6f9f-4904-954c-b853ce89ee06-kube-api-access-7cnt9\") pod \"daea8713-6f9f-4904-954c-b853ce89ee06\" (UID: \"daea8713-6f9f-4904-954c-b853ce89ee06\") "
Jul 16 15:46:11.447965 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:11.447915    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daea8713-6f9f-4904-954c-b853ce89ee06-kube-api-access-7cnt9" (OuterVolumeSpecName: "kube-api-access-7cnt9") pod "daea8713-6f9f-4904-954c-b853ce89ee06" (UID: "daea8713-6f9f-4904-954c-b853ce89ee06"). InnerVolumeSpecName "kube-api-access-7cnt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:46:11.456294 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:11.456270    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/daea8713-6f9f-4904-954c-b853ce89ee06-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "daea8713-6f9f-4904-954c-b853ce89ee06" (UID: "daea8713-6f9f-4904-954c-b853ce89ee06"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 15:46:11.547227 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:11.547136    2570 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/daea8713-6f9f-4904-954c-b853ce89ee06-tls-cert\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:46:11.547227 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:11.547167    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cnt9\" (UniqueName: \"kubernetes.io/projected/daea8713-6f9f-4904-954c-b853ce89ee06-kube-api-access-7cnt9\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:46:12.054655 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:12.054618    2570 generic.go:356] "Generic (PLEG): container finished" podID="daea8713-6f9f-4904-954c-b853ce89ee06" containerID="56d93d9c52c7e12932978b2cdcee561861bb52e527e759a6756bac28ca3fa241" exitCode=0
Jul 16 15:46:12.055094 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:12.054664    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw"
Jul 16 15:46:12.055094 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:12.054672    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw" event={"ID":"daea8713-6f9f-4904-954c-b853ce89ee06","Type":"ContainerDied","Data":"56d93d9c52c7e12932978b2cdcee561861bb52e527e759a6756bac28ca3fa241"}
Jul 16 15:46:12.055094 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:12.054710    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fbb79cdf7-6t5lw" event={"ID":"daea8713-6f9f-4904-954c-b853ce89ee06","Type":"ContainerDied","Data":"9f4810d369386921e3ba0401f3b31ff03082e0aadac18dd9189358d99ee82768"}
Jul 16 15:46:12.055094 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:12.054731    2570 scope.go:117] "RemoveContainer" containerID="56d93d9c52c7e12932978b2cdcee561861bb52e527e759a6756bac28ca3fa241"
Jul 16 15:46:12.063621 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:12.063602    2570 scope.go:117] "RemoveContainer" containerID="56d93d9c52c7e12932978b2cdcee561861bb52e527e759a6756bac28ca3fa241"
Jul 16 15:46:12.063875 ip-10-0-141-84 kubenswrapper[2570]: E0716 15:46:12.063856    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56d93d9c52c7e12932978b2cdcee561861bb52e527e759a6756bac28ca3fa241\": container with ID starting with 56d93d9c52c7e12932978b2cdcee561861bb52e527e759a6756bac28ca3fa241 not found: ID does not exist" containerID="56d93d9c52c7e12932978b2cdcee561861bb52e527e759a6756bac28ca3fa241"
Jul 16 15:46:12.063923 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:12.063885    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56d93d9c52c7e12932978b2cdcee561861bb52e527e759a6756bac28ca3fa241"} err="failed to get container status \"56d93d9c52c7e12932978b2cdcee561861bb52e527e759a6756bac28ca3fa241\": rpc error: code = NotFound desc = could not find container \"56d93d9c52c7e12932978b2cdcee561861bb52e527e759a6756bac28ca3fa241\": container with ID starting with 56d93d9c52c7e12932978b2cdcee561861bb52e527e759a6756bac28ca3fa241 not found: ID does not exist"
Jul 16 15:46:12.075783 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:12.075740    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7fbb79cdf7-6t5lw"]
Jul 16 15:46:12.077961 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:12.077927    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7fbb79cdf7-6t5lw"]
Jul 16 15:46:12.267755 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:46:12.267714    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daea8713-6f9f-4904-954c-b853ce89ee06" path="/var/lib/kubelet/pods/daea8713-6f9f-4904-954c-b853ce89ee06/volumes"
Jul 16 15:47:27.072831 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:27.072739    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:47:27.185300 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:27.185250    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-775f5d4dfb-kzklc"]
Jul 16 15:47:27.185549 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:27.185520    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-775f5d4dfb-kzklc" podUID="927f8abf-4fa8-4d94-8e0a-52b61b3c154f" containerName="manager" containerID="cri-o://5d489251196e6a9c9e061f20b10aa055f5c462b38cd3e0f4d2762d71f93137d6" gracePeriod=30
Jul 16 15:47:27.401701 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:27.401672    2570 generic.go:356] "Generic (PLEG): container finished" podID="927f8abf-4fa8-4d94-8e0a-52b61b3c154f" containerID="5d489251196e6a9c9e061f20b10aa055f5c462b38cd3e0f4d2762d71f93137d6" exitCode=0
Jul 16 15:47:27.401856 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:27.401741    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-775f5d4dfb-kzklc" event={"ID":"927f8abf-4fa8-4d94-8e0a-52b61b3c154f","Type":"ContainerDied","Data":"5d489251196e6a9c9e061f20b10aa055f5c462b38cd3e0f4d2762d71f93137d6"}
Jul 16 15:47:27.441112 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:27.441081    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-775f5d4dfb-kzklc"
Jul 16 15:47:27.515592 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:27.515557    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/927f8abf-4fa8-4d94-8e0a-52b61b3c154f-webhook-cert\") pod \"927f8abf-4fa8-4d94-8e0a-52b61b3c154f\" (UID: \"927f8abf-4fa8-4d94-8e0a-52b61b3c154f\") "
Jul 16 15:47:27.515766 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:27.515640    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zh9z\" (UniqueName: \"kubernetes.io/projected/927f8abf-4fa8-4d94-8e0a-52b61b3c154f-kube-api-access-6zh9z\") pod \"927f8abf-4fa8-4d94-8e0a-52b61b3c154f\" (UID: \"927f8abf-4fa8-4d94-8e0a-52b61b3c154f\") "
Jul 16 15:47:27.517709 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:27.517674    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/927f8abf-4fa8-4d94-8e0a-52b61b3c154f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "927f8abf-4fa8-4d94-8e0a-52b61b3c154f" (UID: "927f8abf-4fa8-4d94-8e0a-52b61b3c154f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 15:47:27.517818 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:27.517683    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/927f8abf-4fa8-4d94-8e0a-52b61b3c154f-kube-api-access-6zh9z" (OuterVolumeSpecName: "kube-api-access-6zh9z") pod "927f8abf-4fa8-4d94-8e0a-52b61b3c154f" (UID: "927f8abf-4fa8-4d94-8e0a-52b61b3c154f"). InnerVolumeSpecName "kube-api-access-6zh9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 15:47:27.616810 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:27.616728    2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/927f8abf-4fa8-4d94-8e0a-52b61b3c154f-webhook-cert\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:47:27.616810 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:27.616757    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zh9z\" (UniqueName: \"kubernetes.io/projected/927f8abf-4fa8-4d94-8e0a-52b61b3c154f-kube-api-access-6zh9z\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 15:47:27.899330 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:27.899244    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:47:28.406947 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:28.406906    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-775f5d4dfb-kzklc"
Jul 16 15:47:28.406947 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:28.406906    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-775f5d4dfb-kzklc" event={"ID":"927f8abf-4fa8-4d94-8e0a-52b61b3c154f","Type":"ContainerDied","Data":"ce832000a230ca084d83b3a83f90bd730b389f9a71d5dada6517860a0819b51f"}
Jul 16 15:47:28.407526 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:28.407016    2570 scope.go:117] "RemoveContainer" containerID="5d489251196e6a9c9e061f20b10aa055f5c462b38cd3e0f4d2762d71f93137d6"
Jul 16 15:47:28.426155 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:28.426130    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-775f5d4dfb-kzklc"]
Jul 16 15:47:28.430296 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:28.430275    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-775f5d4dfb-kzklc"]
Jul 16 15:47:29.207373 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:29.207334    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:47:30.267974 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:47:30.267919    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="927f8abf-4fa8-4d94-8e0a-52b61b3c154f" path="/var/lib/kubelet/pods/927f8abf-4fa8-4d94-8e0a-52b61b3c154f/volumes"
Jul 16 15:48:02.755996 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:02.755960    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:02.853567 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:02.853524    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:03.885773 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:03.885734    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:03.989611 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:03.989578    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:10.726948 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:10.726902    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:10.828139 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:10.828113    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:10.964619 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:10.964579    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:17.923523 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:17.923489    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:18.026619 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:18.026585    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:18.121639 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:18.121604    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:18.227464 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:18.227381    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:18.360404 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:18.360372    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:22.053534 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:22.053483    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:24.947760 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:24.947725    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:25.049000 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:25.048966    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:48:51.299817 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:48:51.299774    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:50:34.199963 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:50:34.199882    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 15:50:34.202266 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:50:34.202245    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 15:54:22.400968 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:54:22.400913    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:55:27.368704 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:55:27.368623    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:55:34.226615 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:55:34.226586    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 15:55:34.228061 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:55:34.228042    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 15:57:59.890959 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:57:59.890859    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:19.892973 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:19.892903    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:28.887469 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:28.887376    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:36.714218 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:36.714181    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:41.868531 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:41.868488    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:45.035968 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:45.035920    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:45.127109 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:45.127076    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:45.572026 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:45.571997    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:51.814124 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:51.814095    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:51.915569 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:51.915529    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:52.030346 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.030314    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"]
Jul 16 15:59:52.030700 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.030687    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daea8713-6f9f-4904-954c-b853ce89ee06" containerName="authorino"
Jul 16 15:59:52.030700 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.030700    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="daea8713-6f9f-4904-954c-b853ce89ee06" containerName="authorino"
Jul 16 15:59:52.030820 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.030710    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="927f8abf-4fa8-4d94-8e0a-52b61b3c154f" containerName="manager"
Jul 16 15:59:52.030820 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.030716    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="927f8abf-4fa8-4d94-8e0a-52b61b3c154f" containerName="manager"
Jul 16 15:59:52.030820 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.030771    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="927f8abf-4fa8-4d94-8e0a-52b61b3c154f" containerName="manager"
Jul 16 15:59:52.030820 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.030779    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="daea8713-6f9f-4904-954c-b853ce89ee06" containerName="authorino"
Jul 16 15:59:52.033748 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.033719    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.036825 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.036807    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9873c762-openshift-default-dockercfg-nkn22\""
Jul 16 15:59:52.060190 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.060163    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"]
Jul 16 15:59:52.065035 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.064964    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:52.159301 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.159270    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:52.168858 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.168837    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flb4s\" (UniqueName: \"kubernetes.io/projected/0aed53c0-a2cb-4f10-af2f-9201537348b2-kube-api-access-flb4s\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.169001 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.168868    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-token\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.169001 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.168890    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-envoy\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.169001 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.168927    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-podinfo\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.169001 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.168966    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-workload-certs\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.169001 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.168981    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aed53c0-a2cb-4f10-af2f-9201537348b2-istiod-ca-cert\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.169185 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.169015    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-workload-socket\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.169185 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.169047    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-credential-socket\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.169185 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.169092    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-data\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.269576 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.269544    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-credential-socket\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.269784 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.269607    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-data\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.269784 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.269641    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flb4s\" (UniqueName: \"kubernetes.io/projected/0aed53c0-a2cb-4f10-af2f-9201537348b2-kube-api-access-flb4s\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.269784 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.269663    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-token\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.269784 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.269683    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-envoy\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.270009 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.269817    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-podinfo\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.270009 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.269865    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-workload-certs\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.270009 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.269891    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aed53c0-a2cb-4f10-af2f-9201537348b2-istiod-ca-cert\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.270009 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.269958    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-workload-socket\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.270251 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.270016    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-credential-socket\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.270251 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.270036    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-data\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.270251 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.270198    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-workload-certs\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.270368 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.270259    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-workload-socket\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.270602 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.270583    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aed53c0-a2cb-4f10-af2f-9201537348b2-istiod-ca-cert\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.272120 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.272101    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-envoy\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.272354 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.272338    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-podinfo\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.279487 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.279462    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-token\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.279644 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.279625    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flb4s\" (UniqueName: \"kubernetes.io/projected/0aed53c0-a2cb-4f10-af2f-9201537348b2-kube-api-access-flb4s\") pod \"e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") " pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.344036 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.344002    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:52.482332 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.482254    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"]
Jul 16 15:59:52.484571 ip-10-0-141-84 kubenswrapper[2570]: W0716 15:59:52.484545    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aed53c0_a2cb_4f10_af2f_9201537348b2.slice/crio-59bd79b5f3c949b62f20e871dfbd1e160a920c04940231ed473e07984d41309b WatchSource:0}: Error finding container 59bd79b5f3c949b62f20e871dfbd1e160a920c04940231ed473e07984d41309b: Status 404 returned error can't find the container with id 59bd79b5f3c949b62f20e871dfbd1e160a920c04940231ed473e07984d41309b
Jul 16 15:59:52.486671 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.486636    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 15:59:52.486766 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.486743    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 15:59:52.486816 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.486798    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 15:59:52.816555 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.816458    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r" event={"ID":"0aed53c0-a2cb-4f10-af2f-9201537348b2","Type":"ContainerStarted","Data":"fc4fc3d22cd82355aaa0a856af8961e9cef243a2bf6afd0369979e97c9eff7de"}
Jul 16 15:59:52.816555 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.816524    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r" event={"ID":"0aed53c0-a2cb-4f10-af2f-9201537348b2","Type":"ContainerStarted","Data":"59bd79b5f3c949b62f20e871dfbd1e160a920c04940231ed473e07984d41309b"}
Jul 16 15:59:52.843717 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:52.843660    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r" podStartSLOduration=0.843645088 podStartE2EDuration="843.645088ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:59:52.84089113 +0000 UTC m=+1459.029755106" watchObservedRunningTime="2026-07-16 15:59:52.843645088 +0000 UTC m=+1459.032509042"
Jul 16 15:59:53.258247 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:53.258211    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:53.345029 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:53.344987    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:53.350196 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:53.350171    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:53.358132 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:53.358109    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:53.482585 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:53.482552    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:53.597182 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:53.597146    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:53.821073 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:53.821047    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:53.822038 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:53.822012    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 15:59:54.097759 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:54.097724    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:54.201088 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:54.201002    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:56.104607 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:56.104575    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:58.412985 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:58.412946    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:58.579959 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:58.579916    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"]
Jul 16 15:59:58.580133 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:58.580110    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r" podUID="0aed53c0-a2cb-4f10-af2f-9201537348b2" containerName="istio-proxy" containerID="cri-o://fc4fc3d22cd82355aaa0a856af8961e9cef243a2bf6afd0369979e97c9eff7de" gracePeriod=30
Jul 16 15:59:58.599387 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:58.599361    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:58.719570 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:58.719466    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 15:59:58.840679 ip-10-0-141-84 kubenswrapper[2570]: I0716 15:59:58.840634    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:01.049951 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:01.049611    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:04.439579 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.439551    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 16:00:04.483526 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.483438    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-envoy\") pod \"0aed53c0-a2cb-4f10-af2f-9201537348b2\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") "
Jul 16 16:00:04.483526 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.483467    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-workload-certs\") pod \"0aed53c0-a2cb-4f10-af2f-9201537348b2\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") "
Jul 16 16:00:04.483526 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.483508    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aed53c0-a2cb-4f10-af2f-9201537348b2-istiod-ca-cert\") pod \"0aed53c0-a2cb-4f10-af2f-9201537348b2\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") "
Jul 16 16:00:04.483804 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.483532    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-token\") pod \"0aed53c0-a2cb-4f10-af2f-9201537348b2\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") "
Jul 16 16:00:04.483804 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.483560    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-podinfo\") pod \"0aed53c0-a2cb-4f10-af2f-9201537348b2\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") "
Jul 16 16:00:04.483804 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.483620    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-data\") pod \"0aed53c0-a2cb-4f10-af2f-9201537348b2\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") "
Jul 16 16:00:04.483804 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.483679    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-workload-socket\") pod \"0aed53c0-a2cb-4f10-af2f-9201537348b2\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") "
Jul 16 16:00:04.483804 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.483723    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-credential-socket\") pod \"0aed53c0-a2cb-4f10-af2f-9201537348b2\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") "
Jul 16 16:00:04.483804 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.483737    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0aed53c0-a2cb-4f10-af2f-9201537348b2" (UID: "0aed53c0-a2cb-4f10-af2f-9201537348b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:04.483804 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.483751    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flb4s\" (UniqueName: \"kubernetes.io/projected/0aed53c0-a2cb-4f10-af2f-9201537348b2-kube-api-access-flb4s\") pod \"0aed53c0-a2cb-4f10-af2f-9201537348b2\" (UID: \"0aed53c0-a2cb-4f10-af2f-9201537348b2\") "
Jul 16 16:00:04.484190 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.483951    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aed53c0-a2cb-4f10-af2f-9201537348b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0aed53c0-a2cb-4f10-af2f-9201537348b2" (UID: "0aed53c0-a2cb-4f10-af2f-9201537348b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 16:00:04.484190 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.484131    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0aed53c0-a2cb-4f10-af2f-9201537348b2" (UID: "0aed53c0-a2cb-4f10-af2f-9201537348b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:04.484190 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.484139    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-workload-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:04.484190 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.484179    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aed53c0-a2cb-4f10-af2f-9201537348b2-istiod-ca-cert\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:04.484400 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.484224    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0aed53c0-a2cb-4f10-af2f-9201537348b2" (UID: "0aed53c0-a2cb-4f10-af2f-9201537348b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:04.484787 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.484763    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "0aed53c0-a2cb-4f10-af2f-9201537348b2" (UID: "0aed53c0-a2cb-4f10-af2f-9201537348b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:04.485853 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.485830    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0aed53c0-a2cb-4f10-af2f-9201537348b2" (UID: "0aed53c0-a2cb-4f10-af2f-9201537348b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:04.486166 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.486140    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "0aed53c0-a2cb-4f10-af2f-9201537348b2" (UID: "0aed53c0-a2cb-4f10-af2f-9201537348b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:00:04.486287 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.486215    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aed53c0-a2cb-4f10-af2f-9201537348b2-kube-api-access-flb4s" (OuterVolumeSpecName: "kube-api-access-flb4s") pod "0aed53c0-a2cb-4f10-af2f-9201537348b2" (UID: "0aed53c0-a2cb-4f10-af2f-9201537348b2"). InnerVolumeSpecName "kube-api-access-flb4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:00:04.486449 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.486428    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0aed53c0-a2cb-4f10-af2f-9201537348b2" (UID: "0aed53c0-a2cb-4f10-af2f-9201537348b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 16:00:04.584984 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.584947    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-data\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:04.584984 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.584977    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-workload-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:04.584984 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.584987    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-credential-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:04.584984 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.584995    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flb4s\" (UniqueName: \"kubernetes.io/projected/0aed53c0-a2cb-4f10-af2f-9201537348b2-kube-api-access-flb4s\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:04.585265 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.585004    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-envoy\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:04.585265 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.585012    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-token\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:04.585265 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.585020    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aed53c0-a2cb-4f10-af2f-9201537348b2-istio-podinfo\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:04.880734 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.880704    2570 generic.go:356] "Generic (PLEG): container finished" podID="0aed53c0-a2cb-4f10-af2f-9201537348b2" containerID="fc4fc3d22cd82355aaa0a856af8961e9cef243a2bf6afd0369979e97c9eff7de" exitCode=0
Jul 16 16:00:04.880913 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.880747    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"
Jul 16 16:00:04.880913 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.880792    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r" event={"ID":"0aed53c0-a2cb-4f10-af2f-9201537348b2","Type":"ContainerDied","Data":"fc4fc3d22cd82355aaa0a856af8961e9cef243a2bf6afd0369979e97c9eff7de"}
Jul 16 16:00:04.880913 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.880850    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r" event={"ID":"0aed53c0-a2cb-4f10-af2f-9201537348b2","Type":"ContainerDied","Data":"59bd79b5f3c949b62f20e871dfbd1e160a920c04940231ed473e07984d41309b"}
Jul 16 16:00:04.880913 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.880879    2570 scope.go:117] "RemoveContainer" containerID="fc4fc3d22cd82355aaa0a856af8961e9cef243a2bf6afd0369979e97c9eff7de"
Jul 16 16:00:04.890456 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.890437    2570 scope.go:117] "RemoveContainer" containerID="fc4fc3d22cd82355aaa0a856af8961e9cef243a2bf6afd0369979e97c9eff7de"
Jul 16 16:00:04.890720 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:00:04.890700    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc4fc3d22cd82355aaa0a856af8961e9cef243a2bf6afd0369979e97c9eff7de\": container with ID starting with fc4fc3d22cd82355aaa0a856af8961e9cef243a2bf6afd0369979e97c9eff7de not found: ID does not exist" containerID="fc4fc3d22cd82355aaa0a856af8961e9cef243a2bf6afd0369979e97c9eff7de"
Jul 16 16:00:04.890765 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.890728    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc4fc3d22cd82355aaa0a856af8961e9cef243a2bf6afd0369979e97c9eff7de"} err="failed to get container status \"fc4fc3d22cd82355aaa0a856af8961e9cef243a2bf6afd0369979e97c9eff7de\": rpc error: code = NotFound desc = could not find container \"fc4fc3d22cd82355aaa0a856af8961e9cef243a2bf6afd0369979e97c9eff7de\": container with ID starting with fc4fc3d22cd82355aaa0a856af8961e9cef243a2bf6afd0369979e97c9eff7de not found: ID does not exist"
Jul 16 16:00:04.907178 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.907148    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"]
Jul 16 16:00:04.914437 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:04.914412    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9873c762-openshift-default-7df9c5449-wfz7r"]
Jul 16 16:00:05.436041 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.436011    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"]
Jul 16 16:00:05.436389 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.436375    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aed53c0-a2cb-4f10-af2f-9201537348b2" containerName="istio-proxy"
Jul 16 16:00:05.436442 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.436391    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aed53c0-a2cb-4f10-af2f-9201537348b2" containerName="istio-proxy"
Jul 16 16:00:05.436478 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.436465    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aed53c0-a2cb-4f10-af2f-9201537348b2" containerName="istio-proxy"
Jul 16 16:00:05.440721 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.440699    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.443863 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.443843    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9717f3de-openshift-default-dockercfg-dk4fr\""
Jul 16 16:00:05.455783 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.455764    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"]
Jul 16 16:00:05.492992 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.492960    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-podinfo\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.493149 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.493004    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-workload-certs\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.493149 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.493031    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-envoy\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.493149 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.493079    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-workload-socket\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.493149 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.493107    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-token\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.493149 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.493121    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7258\" (UniqueName: \"kubernetes.io/projected/3728cccf-32ce-4f49-84bd-1d4a32e1d993-kube-api-access-t7258\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.493149 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.493140    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-credential-socket\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.493342 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.493165    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-data\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.493342 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.493277    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istiod-ca-cert\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.594168 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.594120    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-data\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.594353 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.594219    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istiod-ca-cert\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.594353 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.594289    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-podinfo\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.594353 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.594326    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-workload-certs\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.594502 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.594350    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-envoy\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.594502 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.594381    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-workload-socket\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.594502 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.594413    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-token\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.594502 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.594441    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7258\" (UniqueName: \"kubernetes.io/projected/3728cccf-32ce-4f49-84bd-1d4a32e1d993-kube-api-access-t7258\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.594502 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.594474    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-credential-socket\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.594682 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.594595    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-data\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.594682 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.594638    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-workload-certs\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.594954 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.594910    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istiod-ca-cert\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.595049 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.594985    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-workload-socket\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.595099 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.595041    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-credential-socket\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.596712 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.596675    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-envoy\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.596805 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.596775    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-podinfo\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.607845 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.607824    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7258\" (UniqueName: \"kubernetes.io/projected/3728cccf-32ce-4f49-84bd-1d4a32e1d993-kube-api-access-t7258\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.609012 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.608907    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-token\") pod \"e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") " pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.752284 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.752185    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:05.911492 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.911460    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"]
Jul 16 16:00:05.911724 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:00:05.911702    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3728cccf_32ce_4f49_84bd_1d4a32e1d993.slice/crio-077e67c54f4b071cdf1d798297611b7096393f8012cdb75881b11fe98f04ca6d WatchSource:0}: Error finding container 077e67c54f4b071cdf1d798297611b7096393f8012cdb75881b11fe98f04ca6d: Status 404 returned error can't find the container with id 077e67c54f4b071cdf1d798297611b7096393f8012cdb75881b11fe98f04ca6d
Jul 16 16:00:05.913785 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.913755    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:00:05.913856 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.913814    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:00:05.913856 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:05.913843    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:00:06.054902 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:06.054803    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:06.156138 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:06.156106    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:06.266972 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:06.266917    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aed53c0-a2cb-4f10-af2f-9201537348b2" path="/var/lib/kubelet/pods/0aed53c0-a2cb-4f10-af2f-9201537348b2/volumes"
Jul 16 16:00:06.693174 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:06.693148    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:06.801744 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:06.801713    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:06.887729 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:06.887694    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:06.893576 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:06.893548    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv" event={"ID":"3728cccf-32ce-4f49-84bd-1d4a32e1d993","Type":"ContainerStarted","Data":"48d6a34ba5e7f59508a1df704ce173e02dd43286ac3420d59bea639969dea0e1"}
Jul 16 16:00:06.893699 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:06.893597    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv" event={"ID":"3728cccf-32ce-4f49-84bd-1d4a32e1d993","Type":"ContainerStarted","Data":"077e67c54f4b071cdf1d798297611b7096393f8012cdb75881b11fe98f04ca6d"}
Jul 16 16:00:06.929952 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:06.929886    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv" podStartSLOduration=1.9298673100000001 podStartE2EDuration="1.92986731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:00:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:00:06.928777938 +0000 UTC m=+1473.117641920" watchObservedRunningTime="2026-07-16 16:00:06.92986731 +0000 UTC m=+1473.118731267"
Jul 16 16:00:07.111287 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:07.111254    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:07.652418 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:07.652383    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:07.752301 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:07.752271    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:07.753314 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:07.753287    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:07.757781 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:07.757759    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:07.898310 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:07.898280    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:07.899277 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:07.899253    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:08.765655 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:08.765628    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:11.329344 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:11.329273    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:11.955094 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:11.955059    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:12.616996 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:12.616957    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:17.945735 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:17.945703    2570 generic.go:356] "Generic (PLEG): container finished" podID="3728cccf-32ce-4f49-84bd-1d4a32e1d993" containerID="48d6a34ba5e7f59508a1df704ce173e02dd43286ac3420d59bea639969dea0e1" exitCode=0
Jul 16 16:00:17.946168 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:17.945779    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv" event={"ID":"3728cccf-32ce-4f49-84bd-1d4a32e1d993","Type":"ContainerDied","Data":"48d6a34ba5e7f59508a1df704ce173e02dd43286ac3420d59bea639969dea0e1"}
Jul 16 16:00:17.946211 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:17.946200    2570 scope.go:117] "RemoveContainer" containerID="48d6a34ba5e7f59508a1df704ce173e02dd43286ac3420d59bea639969dea0e1"
Jul 16 16:00:17.947166 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:17.947140    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:00:17.947229 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:17.947203    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:00:17.947269 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:17.947254    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:00:18.201148 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:18.201068    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"]
Jul 16 16:00:18.952250 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:18.952215    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv" event={"ID":"3728cccf-32ce-4f49-84bd-1d4a32e1d993","Type":"ContainerStarted","Data":"6c0407293ff5e61e1190b0ace4928a55661241d8917166f1c1b717a7644f4260"}
Jul 16 16:00:18.952737 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:18.952296    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv" podUID="3728cccf-32ce-4f49-84bd-1d4a32e1d993" containerName="istio-proxy" containerID="cri-o://6c0407293ff5e61e1190b0ace4928a55661241d8917166f1c1b717a7644f4260" gracePeriod=30
Jul 16 16:00:19.540677 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:19.540640    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:19.619354 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:19.619316    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:22.674564 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:22.674510    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:22.898598 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:22.898566    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:24.199620 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.199599    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:24.264079 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.263998    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-workload-socket\") pod \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") "
Jul 16 16:00:24.264079 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264046    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istiod-ca-cert\") pod \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") "
Jul 16 16:00:24.264309 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264087    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7258\" (UniqueName: \"kubernetes.io/projected/3728cccf-32ce-4f49-84bd-1d4a32e1d993-kube-api-access-t7258\") pod \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") "
Jul 16 16:00:24.264309 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264120    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-podinfo\") pod \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") "
Jul 16 16:00:24.264309 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264138    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-workload-certs\") pod \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") "
Jul 16 16:00:24.264309 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264196    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-credential-socket\") pod \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") "
Jul 16 16:00:24.264309 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264221    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-data\") pod \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") "
Jul 16 16:00:24.264309 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264306    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-envoy\") pod \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") "
Jul 16 16:00:24.264615 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264331    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-token\") pod \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\" (UID: \"3728cccf-32ce-4f49-84bd-1d4a32e1d993\") "
Jul 16 16:00:24.264615 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264368    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3728cccf-32ce-4f49-84bd-1d4a32e1d993" (UID: "3728cccf-32ce-4f49-84bd-1d4a32e1d993"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:24.264615 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264535    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3728cccf-32ce-4f49-84bd-1d4a32e1d993" (UID: "3728cccf-32ce-4f49-84bd-1d4a32e1d993"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:24.264615 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264601    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3728cccf-32ce-4f49-84bd-1d4a32e1d993" (UID: "3728cccf-32ce-4f49-84bd-1d4a32e1d993"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 16:00:24.264830 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264806    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-workload-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:24.264887 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264834    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istiod-ca-cert\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:24.264887 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264850    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-workload-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:24.264887 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.264810    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3728cccf-32ce-4f49-84bd-1d4a32e1d993" (UID: "3728cccf-32ce-4f49-84bd-1d4a32e1d993"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:24.265248 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.265215    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-data" (OuterVolumeSpecName: "istio-data") pod "3728cccf-32ce-4f49-84bd-1d4a32e1d993" (UID: "3728cccf-32ce-4f49-84bd-1d4a32e1d993"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:24.266894 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.266861    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3728cccf-32ce-4f49-84bd-1d4a32e1d993" (UID: "3728cccf-32ce-4f49-84bd-1d4a32e1d993"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:24.266894 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.266881    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3728cccf-32ce-4f49-84bd-1d4a32e1d993-kube-api-access-t7258" (OuterVolumeSpecName: "kube-api-access-t7258") pod "3728cccf-32ce-4f49-84bd-1d4a32e1d993" (UID: "3728cccf-32ce-4f49-84bd-1d4a32e1d993"). InnerVolumeSpecName "kube-api-access-t7258". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:00:24.267158 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.267141    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-token" (OuterVolumeSpecName: "istio-token") pod "3728cccf-32ce-4f49-84bd-1d4a32e1d993" (UID: "3728cccf-32ce-4f49-84bd-1d4a32e1d993"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:00:24.267228 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.267162    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3728cccf-32ce-4f49-84bd-1d4a32e1d993" (UID: "3728cccf-32ce-4f49-84bd-1d4a32e1d993"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 16:00:24.366042 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.366012    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7258\" (UniqueName: \"kubernetes.io/projected/3728cccf-32ce-4f49-84bd-1d4a32e1d993-kube-api-access-t7258\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:24.366042 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.366039    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-podinfo\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:24.366042 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.366049    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-credential-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:24.366244 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.366057    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-data\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:24.366244 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.366068    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-envoy\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:24.366244 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.366075    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3728cccf-32ce-4f49-84bd-1d4a32e1d993-istio-token\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:24.837207 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.837175    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:24.925995 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.925962    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:24.982340 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.982304    2570 generic.go:356] "Generic (PLEG): container finished" podID="3728cccf-32ce-4f49-84bd-1d4a32e1d993" containerID="6c0407293ff5e61e1190b0ace4928a55661241d8917166f1c1b717a7644f4260" exitCode=0
Jul 16 16:00:24.982515 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.982361    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv" event={"ID":"3728cccf-32ce-4f49-84bd-1d4a32e1d993","Type":"ContainerDied","Data":"6c0407293ff5e61e1190b0ace4928a55661241d8917166f1c1b717a7644f4260"}
Jul 16 16:00:24.982515 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.982404    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv" event={"ID":"3728cccf-32ce-4f49-84bd-1d4a32e1d993","Type":"ContainerDied","Data":"077e67c54f4b071cdf1d798297611b7096393f8012cdb75881b11fe98f04ca6d"}
Jul 16 16:00:24.982515 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.982426    2570 scope.go:117] "RemoveContainer" containerID="6c0407293ff5e61e1190b0ace4928a55661241d8917166f1c1b717a7644f4260"
Jul 16 16:00:24.982515 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.982362    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"
Jul 16 16:00:24.991190 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.991172    2570 scope.go:117] "RemoveContainer" containerID="48d6a34ba5e7f59508a1df704ce173e02dd43286ac3420d59bea639969dea0e1"
Jul 16 16:00:24.999061 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.999041    2570 scope.go:117] "RemoveContainer" containerID="6c0407293ff5e61e1190b0ace4928a55661241d8917166f1c1b717a7644f4260"
Jul 16 16:00:24.999278 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:00:24.999263    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c0407293ff5e61e1190b0ace4928a55661241d8917166f1c1b717a7644f4260\": container with ID starting with 6c0407293ff5e61e1190b0ace4928a55661241d8917166f1c1b717a7644f4260 not found: ID does not exist" containerID="6c0407293ff5e61e1190b0ace4928a55661241d8917166f1c1b717a7644f4260"
Jul 16 16:00:24.999322 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.999285    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c0407293ff5e61e1190b0ace4928a55661241d8917166f1c1b717a7644f4260"} err="failed to get container status \"6c0407293ff5e61e1190b0ace4928a55661241d8917166f1c1b717a7644f4260\": rpc error: code = NotFound desc = could not find container \"6c0407293ff5e61e1190b0ace4928a55661241d8917166f1c1b717a7644f4260\": container with ID starting with 6c0407293ff5e61e1190b0ace4928a55661241d8917166f1c1b717a7644f4260 not found: ID does not exist"
Jul 16 16:00:24.999322 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.999301    2570 scope.go:117] "RemoveContainer" containerID="48d6a34ba5e7f59508a1df704ce173e02dd43286ac3420d59bea639969dea0e1"
Jul 16 16:00:24.999544 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:00:24.999528    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48d6a34ba5e7f59508a1df704ce173e02dd43286ac3420d59bea639969dea0e1\": container with ID starting with 48d6a34ba5e7f59508a1df704ce173e02dd43286ac3420d59bea639969dea0e1 not found: ID does not exist" containerID="48d6a34ba5e7f59508a1df704ce173e02dd43286ac3420d59bea639969dea0e1"
Jul 16 16:00:24.999589 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:24.999550    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48d6a34ba5e7f59508a1df704ce173e02dd43286ac3420d59bea639969dea0e1"} err="failed to get container status \"48d6a34ba5e7f59508a1df704ce173e02dd43286ac3420d59bea639969dea0e1\": rpc error: code = NotFound desc = could not find container \"48d6a34ba5e7f59508a1df704ce173e02dd43286ac3420d59bea639969dea0e1\": container with ID starting with 48d6a34ba5e7f59508a1df704ce173e02dd43286ac3420d59bea639969dea0e1 not found: ID does not exist"
Jul 16 16:00:25.031173 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.031149    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"]
Jul 16 16:00:25.035816 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.034954    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9717f3de-openshift-default-565755cd87-zfgnv"]
Jul 16 16:00:25.160353 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.160320    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"]
Jul 16 16:00:25.160791 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.160767    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3728cccf-32ce-4f49-84bd-1d4a32e1d993" containerName="istio-proxy"
Jul 16 16:00:25.160791 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.160791    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3728cccf-32ce-4f49-84bd-1d4a32e1d993" containerName="istio-proxy"
Jul 16 16:00:25.160953 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.160806    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3728cccf-32ce-4f49-84bd-1d4a32e1d993" containerName="istio-proxy"
Jul 16 16:00:25.160953 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.160815    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3728cccf-32ce-4f49-84bd-1d4a32e1d993" containerName="istio-proxy"
Jul 16 16:00:25.160953 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.160908    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3728cccf-32ce-4f49-84bd-1d4a32e1d993" containerName="istio-proxy"
Jul 16 16:00:25.165492 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.165475    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.169358 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.169335    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-e590f3b0-openshift-default-dockercfg-sspn7\""
Jul 16 16:00:25.194312 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.194277    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"]
Jul 16 16:00:25.201227 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.201207    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:25.275425 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.275394    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-workload-socket\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.275585 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.275433    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-token\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.275585 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.275489    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-data\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.275585 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.275506    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istiod-ca-cert\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.275712 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.275582    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-workload-certs\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.275712 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.275624    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-podinfo\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.275712 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.275644    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-envoy\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.275712 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.275669    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-credential-socket\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.275712 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.275701    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9kps\" (UniqueName: \"kubernetes.io/projected/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-kube-api-access-l9kps\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.377176 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.377088    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-workload-socket\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.377176 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.377123    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-token\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.377176 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.377148    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-data\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.377176 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.377166    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istiod-ca-cert\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.377489 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.377203    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-workload-certs\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.377489 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.377245    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-podinfo\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.377489 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.377345    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-envoy\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.377489 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.377381    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-credential-socket\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.377489 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.377416    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9kps\" (UniqueName: \"kubernetes.io/projected/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-kube-api-access-l9kps\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.377760 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.377556    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-workload-socket\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.377760 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.377663    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-workload-certs\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.377867 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.377777    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-data\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.377867 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.377817    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-credential-socket\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.377973 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.377915    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istiod-ca-cert\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.379581 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.379559    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-envoy\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.379895 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.379868    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-podinfo\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.404191 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.404168    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-token\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.406110 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.406089    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9kps\" (UniqueName: \"kubernetes.io/projected/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-kube-api-access-l9kps\") pod \"e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") " pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.476167 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.476133    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:25.562194 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.562158    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:25.635082 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.635059    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"]
Jul 16 16:00:25.636564 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:00:25.636528    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f7e77b4_5efa_4821_9d2e_f2caccb1941a.slice/crio-fc1154d10f3abebb4938ab9dd4680d2dd8e5c0ad5dc512974f7b7a14ff79ce88 WatchSource:0}: Error finding container fc1154d10f3abebb4938ab9dd4680d2dd8e5c0ad5dc512974f7b7a14ff79ce88: Status 404 returned error can't find the container with id fc1154d10f3abebb4938ab9dd4680d2dd8e5c0ad5dc512974f7b7a14ff79ce88
Jul 16 16:00:25.638614 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.638582    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:00:25.638686 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.638645    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:00:25.638686 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.638673    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:00:25.644096 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.644076    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:25.989543 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.989451    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v" event={"ID":"6f7e77b4-5efa-4821-9d2e-f2caccb1941a","Type":"ContainerStarted","Data":"dfc2f8fca6c1971c112033b4889b7499ed6d21777042ac7cf25c6fda56946a9f"}
Jul 16 16:00:25.989543 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:25.989499    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v" event={"ID":"6f7e77b4-5efa-4821-9d2e-f2caccb1941a","Type":"ContainerStarted","Data":"fc1154d10f3abebb4938ab9dd4680d2dd8e5c0ad5dc512974f7b7a14ff79ce88"}
Jul 16 16:00:26.019029 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:26.018977    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v" podStartSLOduration=1.018961245 podStartE2EDuration="1.018961245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:00:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:00:26.017501575 +0000 UTC m=+1492.206365575" watchObservedRunningTime="2026-07-16 16:00:26.018961245 +0000 UTC m=+1492.207825261"
Jul 16 16:00:26.268740 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:26.268662    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3728cccf-32ce-4f49-84bd-1d4a32e1d993" path="/var/lib/kubelet/pods/3728cccf-32ce-4f49-84bd-1d4a32e1d993/volumes"
Jul 16 16:00:26.439220 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:26.439183    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:26.476705 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:26.476674    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:26.481378 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:26.481355    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:26.575992 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:26.575889    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:26.765655 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:26.765617    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:26.994296 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:26.994262    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:26.995254 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:26.995237    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:28.482717 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:28.482686    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:28.582406 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:28.582377    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:28.912642 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:28.912611    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:30.801543 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:30.801507    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:30.961415 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:30.961382    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"]
Jul 16 16:00:30.961625 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:30.961601    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v" podUID="6f7e77b4-5efa-4821-9d2e-f2caccb1941a" containerName="istio-proxy" containerID="cri-o://dfc2f8fca6c1971c112033b4889b7499ed6d21777042ac7cf25c6fda56946a9f" gracePeriod=30
Jul 16 16:00:30.976848 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:30.976823    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:31.115916 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:31.115854    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:31.727337 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:31.727311    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:33.498028 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:33.497976    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:34.253020 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:34.252996    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 16:00:34.254595 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:34.254573    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 16:00:36.215375 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.215350    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:36.382658 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.382620    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-token\") pod \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") "
Jul 16 16:00:36.382821 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.382694    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-workload-certs\") pod \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") "
Jul 16 16:00:36.382821 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.382722    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-workload-socket\") pod \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") "
Jul 16 16:00:36.382821 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.382749    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istiod-ca-cert\") pod \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") "
Jul 16 16:00:36.382821 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.382775    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9kps\" (UniqueName: \"kubernetes.io/projected/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-kube-api-access-l9kps\") pod \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") "
Jul 16 16:00:36.382821 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.382819    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-podinfo\") pod \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") "
Jul 16 16:00:36.383138 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.382884    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-credential-socket\") pod \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") "
Jul 16 16:00:36.383138 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.382912    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-data\") pod \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") "
Jul 16 16:00:36.383138 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.382961    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-envoy\") pod \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\" (UID: \"6f7e77b4-5efa-4821-9d2e-f2caccb1941a\") "
Jul 16 16:00:36.383138 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.383008    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6f7e77b4-5efa-4821-9d2e-f2caccb1941a" (UID: "6f7e77b4-5efa-4821-9d2e-f2caccb1941a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:36.383138 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.383119    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6f7e77b4-5efa-4821-9d2e-f2caccb1941a" (UID: "6f7e77b4-5efa-4821-9d2e-f2caccb1941a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:36.383394 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.383163    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6f7e77b4-5efa-4821-9d2e-f2caccb1941a" (UID: "6f7e77b4-5efa-4821-9d2e-f2caccb1941a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:36.383394 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.383222    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-data" (OuterVolumeSpecName: "istio-data") pod "6f7e77b4-5efa-4821-9d2e-f2caccb1941a" (UID: "6f7e77b4-5efa-4821-9d2e-f2caccb1941a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:36.383394 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.383269    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6f7e77b4-5efa-4821-9d2e-f2caccb1941a" (UID: "6f7e77b4-5efa-4821-9d2e-f2caccb1941a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 16:00:36.383394 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.383363    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istiod-ca-cert\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:36.383394 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.383383    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-credential-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:36.383655 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.383399    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-data\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:36.383655 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.383414    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-workload-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:36.383655 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.383429    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-workload-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:36.385236 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.385207    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-token" (OuterVolumeSpecName: "istio-token") pod "6f7e77b4-5efa-4821-9d2e-f2caccb1941a" (UID: "6f7e77b4-5efa-4821-9d2e-f2caccb1941a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:00:36.385236 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.385213    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6f7e77b4-5efa-4821-9d2e-f2caccb1941a" (UID: "6f7e77b4-5efa-4821-9d2e-f2caccb1941a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:00:36.385449 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.385422    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-kube-api-access-l9kps" (OuterVolumeSpecName: "kube-api-access-l9kps") pod "6f7e77b4-5efa-4821-9d2e-f2caccb1941a" (UID: "6f7e77b4-5efa-4821-9d2e-f2caccb1941a"). InnerVolumeSpecName "kube-api-access-l9kps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:00:36.385449 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.385432    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6f7e77b4-5efa-4821-9d2e-f2caccb1941a" (UID: "6f7e77b4-5efa-4821-9d2e-f2caccb1941a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 16:00:36.484163 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.484127    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9kps\" (UniqueName: \"kubernetes.io/projected/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-kube-api-access-l9kps\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:36.484163 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.484158    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-podinfo\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:36.484163 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.484171    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-envoy\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:36.484433 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:36.484182    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7e77b4-5efa-4821-9d2e-f2caccb1941a-istio-token\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:00:37.048642 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:37.048607    2570 generic.go:356] "Generic (PLEG): container finished" podID="6f7e77b4-5efa-4821-9d2e-f2caccb1941a" containerID="dfc2f8fca6c1971c112033b4889b7499ed6d21777042ac7cf25c6fda56946a9f" exitCode=0
Jul 16 16:00:37.048821 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:37.048651    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"
Jul 16 16:00:37.048821 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:37.048694    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v" event={"ID":"6f7e77b4-5efa-4821-9d2e-f2caccb1941a","Type":"ContainerDied","Data":"dfc2f8fca6c1971c112033b4889b7499ed6d21777042ac7cf25c6fda56946a9f"}
Jul 16 16:00:37.048821 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:37.048745    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v" event={"ID":"6f7e77b4-5efa-4821-9d2e-f2caccb1941a","Type":"ContainerDied","Data":"fc1154d10f3abebb4938ab9dd4680d2dd8e5c0ad5dc512974f7b7a14ff79ce88"}
Jul 16 16:00:37.048821 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:37.048769    2570 scope.go:117] "RemoveContainer" containerID="dfc2f8fca6c1971c112033b4889b7499ed6d21777042ac7cf25c6fda56946a9f"
Jul 16 16:00:37.057878 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:37.057860    2570 scope.go:117] "RemoveContainer" containerID="dfc2f8fca6c1971c112033b4889b7499ed6d21777042ac7cf25c6fda56946a9f"
Jul 16 16:00:37.058145 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:00:37.058125    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfc2f8fca6c1971c112033b4889b7499ed6d21777042ac7cf25c6fda56946a9f\": container with ID starting with dfc2f8fca6c1971c112033b4889b7499ed6d21777042ac7cf25c6fda56946a9f not found: ID does not exist" containerID="dfc2f8fca6c1971c112033b4889b7499ed6d21777042ac7cf25c6fda56946a9f"
Jul 16 16:00:37.058234 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:37.058156    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfc2f8fca6c1971c112033b4889b7499ed6d21777042ac7cf25c6fda56946a9f"} err="failed to get container status \"dfc2f8fca6c1971c112033b4889b7499ed6d21777042ac7cf25c6fda56946a9f\": rpc error: code = NotFound desc = could not find container \"dfc2f8fca6c1971c112033b4889b7499ed6d21777042ac7cf25c6fda56946a9f\": container with ID starting with dfc2f8fca6c1971c112033b4889b7499ed6d21777042ac7cf25c6fda56946a9f not found: ID does not exist"
Jul 16 16:00:37.075307 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:37.075283    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"]
Jul 16 16:00:37.084181 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:37.084160    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-e590f3b0-openshift-default-6dfb77fc47-zcc5v"]
Jul 16 16:00:37.841703 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:37.841672    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:37.945597 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:37.945567    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:38.267247 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:38.267163    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f7e77b4-5efa-4821-9d2e-f2caccb1941a" path="/var/lib/kubelet/pods/6f7e77b4-5efa-4821-9d2e-f2caccb1941a/volumes"
Jul 16 16:00:38.508555 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:38.508508    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:38.623078 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:38.623042    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:40.296339 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:40.296297    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:42.999225 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:42.999189    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:46.389819 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:46.389788    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:46.766088 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:46.765999    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:46.862850 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:46.862824    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:00:47.245052 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:00:47.245021    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:01:06.778240 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:01:06.778203    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:01:06.881581 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:01:06.881543    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:01:13.341249 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:01:13.341213    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:01:21.322468 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:01:21.322424    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:01:21.526453 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:01:21.526416    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:01:21.774160 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:01:21.774124    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:01:37.960370 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:01:37.960341    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:01:45.578335 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:01:45.578288    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:01:45.725766 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:01:45.725736    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:01:45.925903 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:01:45.925870    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:01:56.600237 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:01:56.600199    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:01:57.602882 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:01:57.602847    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:04.091912 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:04.091872    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:04.194336 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:04.194292    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:04.294864 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:04.294829    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:04.393142 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:04.393055    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:04.496165 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:04.496130    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:05.441551 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:05.441515    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:05.535124 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:05.535093    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:05.637421 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:05.637383    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:07.753359 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:07.753326    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:14.632846 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:14.632814    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:14.706081 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:14.706053    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:15.405638 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:15.405605    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:23.917172 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:23.917135    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:24.033137 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:24.033074    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:24.258671 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:24.258590    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:25.039195 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.039160    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:25.257403 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.254692    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"]
Jul 16 16:02:25.257403 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.255542    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f7e77b4-5efa-4821-9d2e-f2caccb1941a" containerName="istio-proxy"
Jul 16 16:02:25.257403 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.255564    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f7e77b4-5efa-4821-9d2e-f2caccb1941a" containerName="istio-proxy"
Jul 16 16:02:25.257403 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.255716    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3728cccf-32ce-4f49-84bd-1d4a32e1d993" containerName="istio-proxy"
Jul 16 16:02:25.257403 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.255737    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f7e77b4-5efa-4821-9d2e-f2caccb1941a" containerName="istio-proxy"
Jul 16 16:02:25.259788 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.259760    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.262873 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.262852    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3774490c-openshift-default-dockercfg-rz9tw\""
Jul 16 16:02:25.367661 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.367623    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:25.438791 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.438754    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-workload-certs\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.438981 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.438800    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-workload-socket\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.438981 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.438819    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-data\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.438981 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.438870    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-envoy\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.438981 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.438911    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-token\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.438981 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.438954    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-podinfo\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.439156 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.439019    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-credential-socket\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.439156 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.439037    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6739844c-2043-4871-972c-c1a9c1bf10ab-istiod-ca-cert\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.439156 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.439068    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44ps9\" (UniqueName: \"kubernetes.io/projected/6739844c-2043-4871-972c-c1a9c1bf10ab-kube-api-access-44ps9\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.468060 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.468025    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"]
Jul 16 16:02:25.539588 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.539556    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-credential-socket\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.539588 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.539590    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6739844c-2043-4871-972c-c1a9c1bf10ab-istiod-ca-cert\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.539849 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.539614    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44ps9\" (UniqueName: \"kubernetes.io/projected/6739844c-2043-4871-972c-c1a9c1bf10ab-kube-api-access-44ps9\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.539849 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.539683    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-workload-certs\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.539849 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.539714    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-workload-socket\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.540056 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.539870    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-data\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.540056 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.539921    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-envoy\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.540056 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.539983    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-token\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.540056 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.540010    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-podinfo\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.540268 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.540114    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-workload-socket\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.540268 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.540250    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-workload-certs\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.540406 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.540285    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-data\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.540406 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.540390    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-credential-socket\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.540406 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.540397    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6739844c-2043-4871-972c-c1a9c1bf10ab-istiod-ca-cert\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.542234 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.542216    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-envoy\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.542586 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.542564    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-podinfo\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.573800 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.573775    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44ps9\" (UniqueName: \"kubernetes.io/projected/6739844c-2043-4871-972c-c1a9c1bf10ab-kube-api-access-44ps9\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.578038 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.578011    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-token\") pod \"e2e-mt-3774490c-openshift-default-555875d94c-xvkb4\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") " pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:25.871914 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:25.871878    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:26.062665 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:26.062635    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"]
Jul 16 16:02:26.065709 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:02:26.065681    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6739844c_2043_4871_972c_c1a9c1bf10ab.slice/crio-6c05d03c519d77623367a33c3db1c8d0307684201efe91de3e8d0686efd04add WatchSource:0}: Error finding container 6c05d03c519d77623367a33c3db1c8d0307684201efe91de3e8d0686efd04add: Status 404 returned error can't find the container with id 6c05d03c519d77623367a33c3db1c8d0307684201efe91de3e8d0686efd04add
Jul 16 16:02:26.067922 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:26.067891    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:02:26.068009 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:26.067972    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:02:26.068009 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:26.068003    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:02:26.556839 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:26.556800    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4" event={"ID":"6739844c-2043-4871-972c-c1a9c1bf10ab","Type":"ContainerStarted","Data":"ed2264eda6ed8f9da0469ec71ee011e5026784ccb144fb2fd4f3699e1d7f3114"}
Jul 16 16:02:26.557049 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:26.556856    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4" event={"ID":"6739844c-2043-4871-972c-c1a9c1bf10ab","Type":"ContainerStarted","Data":"6c05d03c519d77623367a33c3db1c8d0307684201efe91de3e8d0686efd04add"}
Jul 16 16:02:26.629664 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:26.629598    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4" podStartSLOduration=1.629579357 podStartE2EDuration="1.629579357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:02:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:02:26.628058971 +0000 UTC m=+1612.816922926" watchObservedRunningTime="2026-07-16 16:02:26.629579357 +0000 UTC m=+1612.818443312"
Jul 16 16:02:26.873065 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:26.873027    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:27.878346 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:27.878310    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:28.566626 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:28.566592    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:28.567552 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:28.567530    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:29.416612 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:29.416584    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:29.635853 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:29.635815    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:29.743435 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:29.743329    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:30.010975 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:30.010860    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:31.456034 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:31.455994    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:31.604559 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:31.604527    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:33.736812 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:33.736756    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:34.552117 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:34.552086    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:34.656743 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:34.656707    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:35.069715 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:35.069679    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:43.998579 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:43.998549    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:44.090231 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:44.090192    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:44.211775 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:44.210788    2570 patch_prober.go:28] interesting pod/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": read tcp 10.133.0.2:44908->10.133.0.48:15021: read: connection reset by peer" start-of-body=
Jul 16 16:02:44.211775 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:44.210874    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4" podUID="6739844c-2043-4871-972c-c1a9c1bf10ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": read tcp 10.133.0.2:44908->10.133.0.48:15021: read: connection reset by peer"
Jul 16 16:02:44.304199 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:44.304164    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:44.641055 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:44.641017    2570 generic.go:356] "Generic (PLEG): container finished" podID="6739844c-2043-4871-972c-c1a9c1bf10ab" containerID="ed2264eda6ed8f9da0469ec71ee011e5026784ccb144fb2fd4f3699e1d7f3114" exitCode=0
Jul 16 16:02:44.641247 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:44.641092    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4" event={"ID":"6739844c-2043-4871-972c-c1a9c1bf10ab","Type":"ContainerDied","Data":"ed2264eda6ed8f9da0469ec71ee011e5026784ccb144fb2fd4f3699e1d7f3114"}
Jul 16 16:02:44.641499 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:44.641486    2570 scope.go:117] "RemoveContainer" containerID="ed2264eda6ed8f9da0469ec71ee011e5026784ccb144fb2fd4f3699e1d7f3114"
Jul 16 16:02:44.642543 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:44.642516    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:02:44.642595 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:44.642585    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:02:44.642658 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:44.642619    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:02:45.651026 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:45.650987    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4" event={"ID":"6739844c-2043-4871-972c-c1a9c1bf10ab","Type":"ContainerStarted","Data":"203dabd74c1ab68b6a59d5129dc84a3058b0dac5d16b7a97dbc8fb82870d2ab8"}
Jul 16 16:02:45.872801 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:45.872766    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:45.873034 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:45.873015    2570 patch_prober.go:28] interesting pod/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 16 16:02:45.873083 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:45.873061    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4" podUID="6739844c-2043-4871-972c-c1a9c1bf10ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 16 16:02:46.872365 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:46.872327    2570 patch_prober.go:28] interesting pod/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 16 16:02:46.872775 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:46.872382    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4" podUID="6739844c-2043-4871-972c-c1a9c1bf10ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 16 16:02:47.872809 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:47.872772    2570 patch_prober.go:28] interesting pod/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 16 16:02:47.873230 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:47.872826    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4" podUID="6739844c-2043-4871-972c-c1a9c1bf10ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 16 16:02:48.873220 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:48.873175    2570 patch_prober.go:28] interesting pod/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 16 16:02:48.873706 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:48.873238    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4" podUID="6739844c-2043-4871-972c-c1a9c1bf10ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 16 16:02:49.875813 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:49.875783    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:49.876248 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:49.876121    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:49.876895 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:49.876878    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:02:52.223214 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:52.223184    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:52.383334 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:52.383296    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:54.230424 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:54.230378    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:54.364158 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:54.364123    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:54.702394 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:54.702347    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"]
Jul 16 16:02:54.702640 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:54.702609    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4" podUID="6739844c-2043-4871-972c-c1a9c1bf10ab" containerName="istio-proxy" containerID="cri-o://203dabd74c1ab68b6a59d5129dc84a3058b0dac5d16b7a97dbc8fb82870d2ab8" gracePeriod=30
Jul 16 16:02:54.729953 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:54.729904    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:55.017047 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:55.016960    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:55.208286 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:55.208252    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:55.428804 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:55.428772    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:55.567705 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:55.567672    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:02:59.958208 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:02:59.958179    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:03:00.063273 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.063188    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-data\") pod \"6739844c-2043-4871-972c-c1a9c1bf10ab\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") "
Jul 16 16:03:00.063273 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.063241    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-envoy\") pod \"6739844c-2043-4871-972c-c1a9c1bf10ab\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") "
Jul 16 16:03:00.063506 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.063278    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44ps9\" (UniqueName: \"kubernetes.io/projected/6739844c-2043-4871-972c-c1a9c1bf10ab-kube-api-access-44ps9\") pod \"6739844c-2043-4871-972c-c1a9c1bf10ab\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") "
Jul 16 16:03:00.063506 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.063322    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-token\") pod \"6739844c-2043-4871-972c-c1a9c1bf10ab\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") "
Jul 16 16:03:00.063506 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.063344    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-podinfo\") pod \"6739844c-2043-4871-972c-c1a9c1bf10ab\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") "
Jul 16 16:03:00.063506 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.063447    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-credential-socket\") pod \"6739844c-2043-4871-972c-c1a9c1bf10ab\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") "
Jul 16 16:03:00.063727 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.063511    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-workload-socket\") pod \"6739844c-2043-4871-972c-c1a9c1bf10ab\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") "
Jul 16 16:03:00.063727 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.063546    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6739844c-2043-4871-972c-c1a9c1bf10ab-istiod-ca-cert\") pod \"6739844c-2043-4871-972c-c1a9c1bf10ab\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") "
Jul 16 16:03:00.063727 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.063594    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-workload-certs\") pod \"6739844c-2043-4871-972c-c1a9c1bf10ab\" (UID: \"6739844c-2043-4871-972c-c1a9c1bf10ab\") "
Jul 16 16:03:00.063947 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.063899    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6739844c-2043-4871-972c-c1a9c1bf10ab" (UID: "6739844c-2043-4871-972c-c1a9c1bf10ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:03:00.064060 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.064029    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6739844c-2043-4871-972c-c1a9c1bf10ab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6739844c-2043-4871-972c-c1a9c1bf10ab" (UID: "6739844c-2043-4871-972c-c1a9c1bf10ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 16:03:00.064346 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.064124    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6739844c-2043-4871-972c-c1a9c1bf10ab" (UID: "6739844c-2043-4871-972c-c1a9c1bf10ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:03:00.064346 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.064149    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6739844c-2043-4871-972c-c1a9c1bf10ab" (UID: "6739844c-2043-4871-972c-c1a9c1bf10ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:03:00.064346 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.064266    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-data" (OuterVolumeSpecName: "istio-data") pod "6739844c-2043-4871-972c-c1a9c1bf10ab" (UID: "6739844c-2043-4871-972c-c1a9c1bf10ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:03:00.065690 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.065662    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6739844c-2043-4871-972c-c1a9c1bf10ab" (UID: "6739844c-2043-4871-972c-c1a9c1bf10ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:03:00.065814 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.065799    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6739844c-2043-4871-972c-c1a9c1bf10ab-kube-api-access-44ps9" (OuterVolumeSpecName: "kube-api-access-44ps9") pod "6739844c-2043-4871-972c-c1a9c1bf10ab" (UID: "6739844c-2043-4871-972c-c1a9c1bf10ab"). InnerVolumeSpecName "kube-api-access-44ps9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:03:00.065923 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.065908    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-token" (OuterVolumeSpecName: "istio-token") pod "6739844c-2043-4871-972c-c1a9c1bf10ab" (UID: "6739844c-2043-4871-972c-c1a9c1bf10ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:03:00.066185 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.066170    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6739844c-2043-4871-972c-c1a9c1bf10ab" (UID: "6739844c-2043-4871-972c-c1a9c1bf10ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 16:03:00.164495 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.164463    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-envoy\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:03:00.164495 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.164491    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44ps9\" (UniqueName: \"kubernetes.io/projected/6739844c-2043-4871-972c-c1a9c1bf10ab-kube-api-access-44ps9\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:03:00.164495 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.164501    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-token\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:03:00.164726 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.164513    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-podinfo\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:03:00.164726 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.164526    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-credential-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:03:00.164726 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.164537    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-workload-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:03:00.164726 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.164549    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6739844c-2043-4871-972c-c1a9c1bf10ab-istiod-ca-cert\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:03:00.164726 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.164558    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-workload-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:03:00.164726 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.164565    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6739844c-2043-4871-972c-c1a9c1bf10ab-istio-data\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:03:00.729006 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.728974    2570 generic.go:356] "Generic (PLEG): container finished" podID="6739844c-2043-4871-972c-c1a9c1bf10ab" containerID="203dabd74c1ab68b6a59d5129dc84a3058b0dac5d16b7a97dbc8fb82870d2ab8" exitCode=0
Jul 16 16:03:00.729193 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.729021    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"
Jul 16 16:03:00.729193 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.729054    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4" event={"ID":"6739844c-2043-4871-972c-c1a9c1bf10ab","Type":"ContainerDied","Data":"203dabd74c1ab68b6a59d5129dc84a3058b0dac5d16b7a97dbc8fb82870d2ab8"}
Jul 16 16:03:00.729193 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.729112    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4" event={"ID":"6739844c-2043-4871-972c-c1a9c1bf10ab","Type":"ContainerDied","Data":"6c05d03c519d77623367a33c3db1c8d0307684201efe91de3e8d0686efd04add"}
Jul 16 16:03:00.729193 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.729147    2570 scope.go:117] "RemoveContainer" containerID="203dabd74c1ab68b6a59d5129dc84a3058b0dac5d16b7a97dbc8fb82870d2ab8"
Jul 16 16:03:00.737722 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.737700    2570 scope.go:117] "RemoveContainer" containerID="ed2264eda6ed8f9da0469ec71ee011e5026784ccb144fb2fd4f3699e1d7f3114"
Jul 16 16:03:00.746065 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.746042    2570 scope.go:117] "RemoveContainer" containerID="203dabd74c1ab68b6a59d5129dc84a3058b0dac5d16b7a97dbc8fb82870d2ab8"
Jul 16 16:03:00.746346 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:03:00.746324    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"203dabd74c1ab68b6a59d5129dc84a3058b0dac5d16b7a97dbc8fb82870d2ab8\": container with ID starting with 203dabd74c1ab68b6a59d5129dc84a3058b0dac5d16b7a97dbc8fb82870d2ab8 not found: ID does not exist" containerID="203dabd74c1ab68b6a59d5129dc84a3058b0dac5d16b7a97dbc8fb82870d2ab8"
Jul 16 16:03:00.746413 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.746362    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203dabd74c1ab68b6a59d5129dc84a3058b0dac5d16b7a97dbc8fb82870d2ab8"} err="failed to get container status \"203dabd74c1ab68b6a59d5129dc84a3058b0dac5d16b7a97dbc8fb82870d2ab8\": rpc error: code = NotFound desc = could not find container \"203dabd74c1ab68b6a59d5129dc84a3058b0dac5d16b7a97dbc8fb82870d2ab8\": container with ID starting with 203dabd74c1ab68b6a59d5129dc84a3058b0dac5d16b7a97dbc8fb82870d2ab8 not found: ID does not exist"
Jul 16 16:03:00.746413 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.746380    2570 scope.go:117] "RemoveContainer" containerID="ed2264eda6ed8f9da0469ec71ee011e5026784ccb144fb2fd4f3699e1d7f3114"
Jul 16 16:03:00.746623 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:03:00.746607    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed2264eda6ed8f9da0469ec71ee011e5026784ccb144fb2fd4f3699e1d7f3114\": container with ID starting with ed2264eda6ed8f9da0469ec71ee011e5026784ccb144fb2fd4f3699e1d7f3114 not found: ID does not exist" containerID="ed2264eda6ed8f9da0469ec71ee011e5026784ccb144fb2fd4f3699e1d7f3114"
Jul 16 16:03:00.746661 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.746626    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed2264eda6ed8f9da0469ec71ee011e5026784ccb144fb2fd4f3699e1d7f3114"} err="failed to get container status \"ed2264eda6ed8f9da0469ec71ee011e5026784ccb144fb2fd4f3699e1d7f3114\": rpc error: code = NotFound desc = could not find container \"ed2264eda6ed8f9da0469ec71ee011e5026784ccb144fb2fd4f3699e1d7f3114\": container with ID starting with ed2264eda6ed8f9da0469ec71ee011e5026784ccb144fb2fd4f3699e1d7f3114 not found: ID does not exist"
Jul 16 16:03:00.771038 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.771002    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"]
Jul 16 16:03:00.777481 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:00.777453    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3774490c-openshift-default-555875d94c-xvkb4"]
Jul 16 16:03:02.267703 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:02.267669    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6739844c-2043-4871-972c-c1a9c1bf10ab" path="/var/lib/kubelet/pods/6739844c-2043-4871-972c-c1a9c1bf10ab/volumes"
Jul 16 16:03:03.295124 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:03.295089    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:03.494323 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:03.494292    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:03.647496 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:03.647460    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:03.802776 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:03.802735    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:11.195221 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:11.195185    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:11.293058 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:11.293028    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:21.012471 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.012428    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:21.611577 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.611543    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"]
Jul 16 16:03:21.612208 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.612185    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6739844c-2043-4871-972c-c1a9c1bf10ab" containerName="istio-proxy"
Jul 16 16:03:21.612208 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.612208    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6739844c-2043-4871-972c-c1a9c1bf10ab" containerName="istio-proxy"
Jul 16 16:03:21.612374 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.612352    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6739844c-2043-4871-972c-c1a9c1bf10ab" containerName="istio-proxy"
Jul 16 16:03:21.615528 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.615508    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.621381 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:03:21.621340    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.639014 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:03:21.638976    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"e2e-mt-ef9fd84b-openshift-default-dockercfg-bkpq6\" is forbidden: User \"system:node:ip-10-0-141-84.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-84.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"e2e-mt-ef9fd84b-openshift-default-dockercfg-bkpq6\"" type="*v1.Secret"
Jul 16 16:03:21.646564 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.646537    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-token\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.646720 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.646575    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-workload-socket\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.646720 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.646605    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istiod-ca-cert\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.646720 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.646626    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqksr\" (UniqueName: \"kubernetes.io/projected/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-kube-api-access-bqksr\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.646720 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.646661    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-data\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.646720 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.646678    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-envoy\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.646927 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.646721    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-workload-certs\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.646927 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.646746    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-podinfo\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.646927 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.646794    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-credential-socket\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.655453 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.655423    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:21.677025 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.676988    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"]
Jul 16 16:03:21.747746 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.747710    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-credential-socket\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.747972 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.747802    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-token\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.747972 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.747832    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-workload-socket\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.747972 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.747873    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istiod-ca-cert\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.747972 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.747902    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqksr\" (UniqueName: \"kubernetes.io/projected/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-kube-api-access-bqksr\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.748259 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.747973    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-data\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.748259 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.748000    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-envoy\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.748259 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.748033    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-workload-certs\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.748259 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.748061    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-podinfo\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.748259 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.748176    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-credential-socket\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.748518 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.748443    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-data\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.748518 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.748227    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-workload-socket\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.748706 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.748682    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-workload-certs\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.748872 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.748800    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istiod-ca-cert\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.750678 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.750653    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-envoy\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.750826 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.750808    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-podinfo\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.774995 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.774966    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-token\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.775125 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.775106    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqksr\" (UniqueName: \"kubernetes.io/projected/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-kube-api-access-bqksr\") pod \"e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") " pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:21.988279 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:21.988191    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:22.549324 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:22.549294    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ef9fd84b-openshift-default-dockercfg-bkpq6\""
Jul 16 16:03:22.556967 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:22.556941    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:22.734183 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:22.734122    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"]
Jul 16 16:03:22.736502 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:03:22.736479    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab76c4c_0e0a_4a36_bdac_b8463fe6fc61.slice/crio-f2d5dee48fb8ddd5672b8f5bf5eecd974252d2f46b2b3224ab15f8cc24622a39 WatchSource:0}: Error finding container f2d5dee48fb8ddd5672b8f5bf5eecd974252d2f46b2b3224ab15f8cc24622a39: Status 404 returned error can't find the container with id f2d5dee48fb8ddd5672b8f5bf5eecd974252d2f46b2b3224ab15f8cc24622a39
Jul 16 16:03:22.738517 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:22.738486    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:03:22.738573 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:22.738556    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:03:22.738612 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:22.738589    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:03:22.829786 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:22.829753    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9" event={"ID":"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61","Type":"ContainerStarted","Data":"7fd95cfcb27e004c78e97ca1cf0f035ec4285cd5d92d7409b8bdcdb6d7a6ffac"}
Jul 16 16:03:22.829960 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:22.829806    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9" event={"ID":"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61","Type":"ContainerStarted","Data":"f2d5dee48fb8ddd5672b8f5bf5eecd974252d2f46b2b3224ab15f8cc24622a39"}
Jul 16 16:03:22.859089 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:22.859033    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9" podStartSLOduration=1.859011451 podStartE2EDuration="1.859011451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:03:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:03:22.856735937 +0000 UTC m=+1669.045599905" watchObservedRunningTime="2026-07-16 16:03:22.859011451 +0000 UTC m=+1669.047875408"
Jul 16 16:03:23.558004 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:23.557967    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:23.562968 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:23.562923    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:23.838137 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:23.838113    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:23.839144 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:23.839123    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:25.218924 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:25.218888    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:25.398845 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:25.398808    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:25.633561 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:25.633520    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:25.785384 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:25.785337    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:26.943416 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:26.943384    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:27.090180 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:27.090151    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:29.627240 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:29.627204    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:38.839864 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:38.839823    2570 patch_prober.go:28] interesting pod/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 16 16:03:38.840333 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:38.839895    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 16 16:03:38.910485 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:38.910442    2570 generic.go:356] "Generic (PLEG): container finished" podID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerID="7fd95cfcb27e004c78e97ca1cf0f035ec4285cd5d92d7409b8bdcdb6d7a6ffac" exitCode=0
Jul 16 16:03:38.910659 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:38.910513    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9" event={"ID":"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61","Type":"ContainerDied","Data":"7fd95cfcb27e004c78e97ca1cf0f035ec4285cd5d92d7409b8bdcdb6d7a6ffac"}
Jul 16 16:03:38.910895 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:38.910882    2570 scope.go:117] "RemoveContainer" containerID="7fd95cfcb27e004c78e97ca1cf0f035ec4285cd5d92d7409b8bdcdb6d7a6ffac"
Jul 16 16:03:38.911921 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:38.911890    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:03:38.912060 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:38.911977    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:03:38.912060 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:38.912009    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:03:39.917766 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:39.917725    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9" event={"ID":"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61","Type":"ContainerStarted","Data":"8c6895be41b6ca7d77086483dddb92bba9ea5dcb1542d20b3ab6cd691af75d25"}
Jul 16 16:03:40.558075 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:40.558038    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:40.558332 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:40.558309    2570 patch_prober.go:28] interesting pod/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 16 16:03:40.558382 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:40.558360    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 16 16:03:41.558399 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:41.558356    2570 patch_prober.go:28] interesting pod/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 16 16:03:41.558831 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:41.558420    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 16 16:03:42.557839 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:42.557798    2570 patch_prober.go:28] interesting pod/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 16 16:03:42.558060 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:42.557874    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 16 16:03:43.557589 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:43.557550    2570 patch_prober.go:28] interesting pod/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 16 16:03:43.558017 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:43.557616    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 16 16:03:44.562026 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:44.561995    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:44.562511 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:44.562368    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:44.562970 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:44.562951    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:03:48.922573 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:48.922540    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"]
Jul 16 16:03:48.922960 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:48.922900    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6739844c-2043-4871-972c-c1a9c1bf10ab" containerName="istio-proxy"
Jul 16 16:03:48.922960 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:48.922912    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6739844c-2043-4871-972c-c1a9c1bf10ab" containerName="istio-proxy"
Jul 16 16:03:48.923040 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:48.923013    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6739844c-2043-4871-972c-c1a9c1bf10ab" containerName="istio-proxy"
Jul 16 16:03:48.926403 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:48.926383    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:48.929215 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:48.929194    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ef9fd84b\"/\"openshift-service-ca.crt\""
Jul 16 16:03:48.930129 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:48.930107    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ef9fd84b\"/\"kube-root-ca.crt\""
Jul 16 16:03:48.930228 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:48.930210    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ef9fd84b\"/\"default-dockercfg-64nb7\""
Jul 16 16:03:48.930280 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:48.930267    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ef9fd84b\"/\"e2e-lifecycle-model-ef9fd84b-kserve-self-signed-certs\""
Jul 16 16:03:48.937980 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:48.937956    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"]
Jul 16 16:03:48.953487 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:48.953464    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:03:49.010339 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.010307    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-model-cache\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.010339 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.010342    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vghkf\" (UniqueName: \"kubernetes.io/projected/69e017a2-eb04-4d60-9362-c75669c79b69-kube-api-access-vghkf\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.010573 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.010419    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-tmp-dir\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.010573 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.010461    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-dshm\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.010573 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.010507    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-home\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.010573 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.010522    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69e017a2-eb04-4d60-9362-c75669c79b69-tls-certs\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.111804 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.111771    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-model-cache\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.111804 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.111808    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vghkf\" (UniqueName: \"kubernetes.io/projected/69e017a2-eb04-4d60-9362-c75669c79b69-kube-api-access-vghkf\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.112110 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.111849    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-tmp-dir\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.112110 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.111873    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-dshm\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.112110 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.112039    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-home\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.112110 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.112071    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69e017a2-eb04-4d60-9362-c75669c79b69-tls-certs\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.112297 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.112273    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-model-cache\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.112360 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.112297    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-tmp-dir\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.112360 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.112347    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-home\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.114371 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.114347    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-dshm\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.114767 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.114736    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69e017a2-eb04-4d60-9362-c75669c79b69-tls-certs\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.121869 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.121834    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vghkf\" (UniqueName: \"kubernetes.io/projected/69e017a2-eb04-4d60-9362-c75669c79b69-kube-api-access-vghkf\") pod \"e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") " pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.235485 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.235391    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:49.369092 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.369066    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"]
Jul 16 16:03:49.371258 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:03:49.371232    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69e017a2_eb04_4d60_9362_c75669c79b69.slice/crio-1899bb4e5be402bcea9547a9163662f61ce138c16580225e879a5e6dbe7d37e6 WatchSource:0}: Error finding container 1899bb4e5be402bcea9547a9163662f61ce138c16580225e879a5e6dbe7d37e6: Status 404 returned error can't find the container with id 1899bb4e5be402bcea9547a9163662f61ce138c16580225e879a5e6dbe7d37e6
Jul 16 16:03:49.980976 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.980920    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p" event={"ID":"69e017a2-eb04-4d60-9362-c75669c79b69","Type":"ContainerStarted","Data":"1d8cca16822e251a743ab58e863324c8bef0b5df14bf5df6f42a9c8460f18a3b"}
Jul 16 16:03:49.981387 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:49.980986    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p" event={"ID":"69e017a2-eb04-4d60-9362-c75669c79b69","Type":"ContainerStarted","Data":"1899bb4e5be402bcea9547a9163662f61ce138c16580225e879a5e6dbe7d37e6"}
Jul 16 16:03:50.002504 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:50.002454    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p" podStartSLOduration=2.002436766 podStartE2EDuration="2.002436766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:03:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:03:49.999390603 +0000 UTC m=+1696.188254557" watchObservedRunningTime="2026-07-16 16:03:50.002436766 +0000 UTC m=+1696.191300771"
Jul 16 16:03:59.235909 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:59.235814    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:59.235909 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:59.235865    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:03:59.248362 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:03:59.248340    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:04:00.037163 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:00.037133    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:04:18.023248 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:18.023213    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:04:18.829002 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:18.828968    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:04:24.356282 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:24.356250    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:04:29.337707 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:29.337673    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"]
Jul 16 16:04:29.437787 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:29.437758    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:04:44.357855 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.357812    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p" podUID="69e017a2-eb04-4d60-9362-c75669c79b69" containerName="main" containerID="cri-o://1d8cca16822e251a743ab58e863324c8bef0b5df14bf5df6f42a9c8460f18a3b" gracePeriod=45
Jul 16 16:04:44.606068 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.606044    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:04:44.700707 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.700622    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-dshm\") pod \"69e017a2-eb04-4d60-9362-c75669c79b69\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") "
Jul 16 16:04:44.700707 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.700700    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-tmp-dir\") pod \"69e017a2-eb04-4d60-9362-c75669c79b69\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") "
Jul 16 16:04:44.700950 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.700745    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vghkf\" (UniqueName: \"kubernetes.io/projected/69e017a2-eb04-4d60-9362-c75669c79b69-kube-api-access-vghkf\") pod \"69e017a2-eb04-4d60-9362-c75669c79b69\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") "
Jul 16 16:04:44.700950 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.700785    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69e017a2-eb04-4d60-9362-c75669c79b69-tls-certs\") pod \"69e017a2-eb04-4d60-9362-c75669c79b69\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") "
Jul 16 16:04:44.700950 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.700817    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-model-cache\") pod \"69e017a2-eb04-4d60-9362-c75669c79b69\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") "
Jul 16 16:04:44.700950 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.700913    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-home\") pod \"69e017a2-eb04-4d60-9362-c75669c79b69\" (UID: \"69e017a2-eb04-4d60-9362-c75669c79b69\") "
Jul 16 16:04:44.701163 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.700999    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "69e017a2-eb04-4d60-9362-c75669c79b69" (UID: "69e017a2-eb04-4d60-9362-c75669c79b69"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:04:44.701221 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.701197    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-tmp-dir\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:04:44.701272 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.701209    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-model-cache" (OuterVolumeSpecName: "model-cache") pod "69e017a2-eb04-4d60-9362-c75669c79b69" (UID: "69e017a2-eb04-4d60-9362-c75669c79b69"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:04:44.701272 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.701220    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-home" (OuterVolumeSpecName: "home") pod "69e017a2-eb04-4d60-9362-c75669c79b69" (UID: "69e017a2-eb04-4d60-9362-c75669c79b69"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:04:44.702908 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.702880    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-dshm" (OuterVolumeSpecName: "dshm") pod "69e017a2-eb04-4d60-9362-c75669c79b69" (UID: "69e017a2-eb04-4d60-9362-c75669c79b69"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:04:44.703026 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.702952    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69e017a2-eb04-4d60-9362-c75669c79b69-kube-api-access-vghkf" (OuterVolumeSpecName: "kube-api-access-vghkf") pod "69e017a2-eb04-4d60-9362-c75669c79b69" (UID: "69e017a2-eb04-4d60-9362-c75669c79b69"). InnerVolumeSpecName "kube-api-access-vghkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:04:44.703026 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.702986    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69e017a2-eb04-4d60-9362-c75669c79b69-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "69e017a2-eb04-4d60-9362-c75669c79b69" (UID: "69e017a2-eb04-4d60-9362-c75669c79b69"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 16:04:44.802009 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.801973    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-dshm\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:04:44.802009 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.802004    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vghkf\" (UniqueName: \"kubernetes.io/projected/69e017a2-eb04-4d60-9362-c75669c79b69-kube-api-access-vghkf\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:04:44.802227 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.802020    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69e017a2-eb04-4d60-9362-c75669c79b69-tls-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:04:44.802227 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.802032    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-model-cache\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:04:44.802227 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:44.802041    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69e017a2-eb04-4d60-9362-c75669c79b69-home\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:04:45.234114 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:45.234074    2570 generic.go:356] "Generic (PLEG): container finished" podID="69e017a2-eb04-4d60-9362-c75669c79b69" containerID="1d8cca16822e251a743ab58e863324c8bef0b5df14bf5df6f42a9c8460f18a3b" exitCode=0
Jul 16 16:04:45.234114 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:45.234112    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"
Jul 16 16:04:45.234402 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:45.234159    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p" event={"ID":"69e017a2-eb04-4d60-9362-c75669c79b69","Type":"ContainerDied","Data":"1d8cca16822e251a743ab58e863324c8bef0b5df14bf5df6f42a9c8460f18a3b"}
Jul 16 16:04:45.234402 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:45.234207    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p" event={"ID":"69e017a2-eb04-4d60-9362-c75669c79b69","Type":"ContainerDied","Data":"1899bb4e5be402bcea9547a9163662f61ce138c16580225e879a5e6dbe7d37e6"}
Jul 16 16:04:45.234402 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:45.234230    2570 scope.go:117] "RemoveContainer" containerID="1d8cca16822e251a743ab58e863324c8bef0b5df14bf5df6f42a9c8460f18a3b"
Jul 16 16:04:45.242912 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:45.242894    2570 scope.go:117] "RemoveContainer" containerID="1d8cca16822e251a743ab58e863324c8bef0b5df14bf5df6f42a9c8460f18a3b"
Jul 16 16:04:45.243200 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:04:45.243182    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d8cca16822e251a743ab58e863324c8bef0b5df14bf5df6f42a9c8460f18a3b\": container with ID starting with 1d8cca16822e251a743ab58e863324c8bef0b5df14bf5df6f42a9c8460f18a3b not found: ID does not exist" containerID="1d8cca16822e251a743ab58e863324c8bef0b5df14bf5df6f42a9c8460f18a3b"
Jul 16 16:04:45.243251 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:45.243210    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d8cca16822e251a743ab58e863324c8bef0b5df14bf5df6f42a9c8460f18a3b"} err="failed to get container status \"1d8cca16822e251a743ab58e863324c8bef0b5df14bf5df6f42a9c8460f18a3b\": rpc error: code = NotFound desc = could not find container \"1d8cca16822e251a743ab58e863324c8bef0b5df14bf5df6f42a9c8460f18a3b\": container with ID starting with 1d8cca16822e251a743ab58e863324c8bef0b5df14bf5df6f42a9c8460f18a3b not found: ID does not exist"
Jul 16 16:04:45.255794 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:45.255769    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"]
Jul 16 16:04:45.260544 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:45.260516    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ef9fd84b/e2e-lifecycle-model-ef9fd84b-kserve-8cc458699-z5w4p"]
Jul 16 16:04:46.266367 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:04:46.266336    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69e017a2-eb04-4d60-9362-c75669c79b69" path="/var/lib/kubelet/pods/69e017a2-eb04-4d60-9362-c75669c79b69/volumes"
Jul 16 16:05:05.642109 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:05.642077    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:08.632558 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:08.632518    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:12.784692 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:12.784657    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:12.838155 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:12.838120    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:13.069909 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:13.069822    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"]
Jul 16 16:05:13.070138 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:13.070111    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerName="istio-proxy" containerID="cri-o://8c6895be41b6ca7d77086483dddb92bba9ea5dcb1542d20b3ab6cd691af75d25" gracePeriod=30
Jul 16 16:05:13.085986 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:13.085957    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:13.517417 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:13.517387    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:14.457647 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:14.457613    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:14.556913 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:14.556882    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:14.564485 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:14.564455    2570 patch_prober.go:28] interesting pod/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 16:05:14.564614 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:14.564512    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 16:05:14.659218 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:14.659184    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:15.464499 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:15.464465    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:15.517717 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:15.517687    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:15.590862 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:15.590827    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:17.163987 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:17.163951    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:18.341213 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.341189    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:05:18.390799 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.390766    2570 generic.go:356] "Generic (PLEG): container finished" podID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerID="8c6895be41b6ca7d77086483dddb92bba9ea5dcb1542d20b3ab6cd691af75d25" exitCode=0
Jul 16 16:05:18.391003 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.390821    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"
Jul 16 16:05:18.391003 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.390851    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9" event={"ID":"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61","Type":"ContainerDied","Data":"8c6895be41b6ca7d77086483dddb92bba9ea5dcb1542d20b3ab6cd691af75d25"}
Jul 16 16:05:18.391003 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.390909    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9" event={"ID":"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61","Type":"ContainerDied","Data":"f2d5dee48fb8ddd5672b8f5bf5eecd974252d2f46b2b3224ab15f8cc24622a39"}
Jul 16 16:05:18.391003 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.390953    2570 scope.go:117] "RemoveContainer" containerID="8c6895be41b6ca7d77086483dddb92bba9ea5dcb1542d20b3ab6cd691af75d25"
Jul 16 16:05:18.403412 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.403387    2570 scope.go:117] "RemoveContainer" containerID="7fd95cfcb27e004c78e97ca1cf0f035ec4285cd5d92d7409b8bdcdb6d7a6ffac"
Jul 16 16:05:18.405662 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.405640    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-data\") pod \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") "
Jul 16 16:05:18.405748 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.405690    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqksr\" (UniqueName: \"kubernetes.io/projected/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-kube-api-access-bqksr\") pod \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") "
Jul 16 16:05:18.405748 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.405716    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-podinfo\") pod \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") "
Jul 16 16:05:18.405830 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.405746    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-workload-socket\") pod \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") "
Jul 16 16:05:18.405830 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.405778    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-envoy\") pod \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") "
Jul 16 16:05:18.405971 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.405875    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-workload-certs\") pod \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") "
Jul 16 16:05:18.406036 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.405968    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istiod-ca-cert\") pod \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") "
Jul 16 16:05:18.406036 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.406007    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-token\") pod \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") "
Jul 16 16:05:18.406143 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.406046    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-credential-socket\") pod \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\" (UID: \"dab76c4c-0e0a-4a36-bdac-b8463fe6fc61\") "
Jul 16 16:05:18.406143 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.406043    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" (UID: "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:05:18.406619 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.406310    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" (UID: "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:05:18.406619 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.406335    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-workload-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:05:18.406619 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.406377    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" (UID: "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 16:05:18.406619 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.406571    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-data" (OuterVolumeSpecName: "istio-data") pod "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" (UID: "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:05:18.406619 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.406594    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" (UID: "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:05:18.407988 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.407961    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-kube-api-access-bqksr" (OuterVolumeSpecName: "kube-api-access-bqksr") pod "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" (UID: "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61"). InnerVolumeSpecName "kube-api-access-bqksr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:05:18.408388 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.408363    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" (UID: "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:05:18.408473 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.408399    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" (UID: "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 16:05:18.408532 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.408511    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-token" (OuterVolumeSpecName: "istio-token") pod "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" (UID: "dab76c4c-0e0a-4a36-bdac-b8463fe6fc61"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:05:18.426201 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.426176    2570 scope.go:117] "RemoveContainer" containerID="8c6895be41b6ca7d77086483dddb92bba9ea5dcb1542d20b3ab6cd691af75d25"
Jul 16 16:05:18.426562 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:05:18.426541    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c6895be41b6ca7d77086483dddb92bba9ea5dcb1542d20b3ab6cd691af75d25\": container with ID starting with 8c6895be41b6ca7d77086483dddb92bba9ea5dcb1542d20b3ab6cd691af75d25 not found: ID does not exist" containerID="8c6895be41b6ca7d77086483dddb92bba9ea5dcb1542d20b3ab6cd691af75d25"
Jul 16 16:05:18.426624 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.426573    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c6895be41b6ca7d77086483dddb92bba9ea5dcb1542d20b3ab6cd691af75d25"} err="failed to get container status \"8c6895be41b6ca7d77086483dddb92bba9ea5dcb1542d20b3ab6cd691af75d25\": rpc error: code = NotFound desc = could not find container \"8c6895be41b6ca7d77086483dddb92bba9ea5dcb1542d20b3ab6cd691af75d25\": container with ID starting with 8c6895be41b6ca7d77086483dddb92bba9ea5dcb1542d20b3ab6cd691af75d25 not found: ID does not exist"
Jul 16 16:05:18.426624 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.426592    2570 scope.go:117] "RemoveContainer" containerID="7fd95cfcb27e004c78e97ca1cf0f035ec4285cd5d92d7409b8bdcdb6d7a6ffac"
Jul 16 16:05:18.426863 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:05:18.426837    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fd95cfcb27e004c78e97ca1cf0f035ec4285cd5d92d7409b8bdcdb6d7a6ffac\": container with ID starting with 7fd95cfcb27e004c78e97ca1cf0f035ec4285cd5d92d7409b8bdcdb6d7a6ffac not found: ID does not exist" containerID="7fd95cfcb27e004c78e97ca1cf0f035ec4285cd5d92d7409b8bdcdb6d7a6ffac"
Jul 16 16:05:18.426974 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.426867    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fd95cfcb27e004c78e97ca1cf0f035ec4285cd5d92d7409b8bdcdb6d7a6ffac"} err="failed to get container status \"7fd95cfcb27e004c78e97ca1cf0f035ec4285cd5d92d7409b8bdcdb6d7a6ffac\": rpc error: code = NotFound desc = could not find container \"7fd95cfcb27e004c78e97ca1cf0f035ec4285cd5d92d7409b8bdcdb6d7a6ffac\": container with ID starting with 7fd95cfcb27e004c78e97ca1cf0f035ec4285cd5d92d7409b8bdcdb6d7a6ffac not found: ID does not exist"
Jul 16 16:05:18.507529 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.507490    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqksr\" (UniqueName: \"kubernetes.io/projected/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-kube-api-access-bqksr\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:05:18.507529 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.507523    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-podinfo\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:05:18.507529 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.507532    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-envoy\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:05:18.507749 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.507541    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-workload-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:05:18.507749 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.507551    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istiod-ca-cert\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:05:18.507749 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.507560    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-token\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:05:18.507749 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.507567    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-credential-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:05:18.507749 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.507575    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61-istio-data\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:05:18.713310 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.713275    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"]
Jul 16 16:05:18.718089 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:18.718047    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ef9fd84b-openshift-default-7cd954795f-d5xx9"]
Jul 16 16:05:20.268689 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:20.268648    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" path="/var/lib/kubelet/pods/dab76c4c-0e0a-4a36-bdac-b8463fe6fc61/volumes"
Jul 16 16:05:23.090465 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:23.090431    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:23.142201 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:23.142174    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:23.842106 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:23.842070    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:29.979517 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:29.979435    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:30.084078 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:30.084043    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:30.182654 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:30.182624    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:34.285070 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:34.285040    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 16:05:34.288081 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:34.288063    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 16:05:40.744834 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:40.744796    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:05:41.108310 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:05:41.108277    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:02.119476 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:02.119439    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:08.056675 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.056643    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:08.158507 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.158469    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:08.438744 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.438709    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:08.840236 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.840203    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"]
Jul 16 16:06:08.840714 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.840700    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerName="istio-proxy"
Jul 16 16:06:08.840760 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.840717    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerName="istio-proxy"
Jul 16 16:06:08.840760 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.840736    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69e017a2-eb04-4d60-9362-c75669c79b69" containerName="main"
Jul 16 16:06:08.840760 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.840746    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="69e017a2-eb04-4d60-9362-c75669c79b69" containerName="main"
Jul 16 16:06:08.840858 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.840839    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerName="istio-proxy"
Jul 16 16:06:08.840858 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.840851    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerName="istio-proxy"
Jul 16 16:06:08.840920 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.840862    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="69e017a2-eb04-4d60-9362-c75669c79b69" containerName="main"
Jul 16 16:06:08.844176 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.844150    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:08.846728 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.846705    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:08.847590 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.847571    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-b8455d-openshift-default-dockercfg-jktkj\""
Jul 16 16:06:08.862468 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.862442    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"]
Jul 16 16:06:08.940363 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.940328    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:08.952718 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.952676    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-workload-certs\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:08.952718 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.952724    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-data\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:08.952918 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.952795    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-token\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:08.952918 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.952831    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm8nd\" (UniqueName: \"kubernetes.io/projected/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-kube-api-access-rm8nd\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:08.952918 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.952855    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-credential-socket\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:08.953033 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.952976    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-envoy\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:08.953033 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.953001    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istiod-ca-cert\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:08.953100 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.953030    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-podinfo\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:08.953100 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:08.953055    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-workload-socket\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.053745 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.053701    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-credential-socket\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.053966 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.053863    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-envoy\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.053966 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.053902    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istiod-ca-cert\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.053966 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.053955    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-podinfo\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.054146 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.053981    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-workload-socket\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.054146 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.054028    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-workload-certs\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.054146 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.054056    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-data\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.054146 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.054102    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-token\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.054146 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.054135    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm8nd\" (UniqueName: \"kubernetes.io/projected/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-kube-api-access-rm8nd\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.054416 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.054194    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-credential-socket\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.054416 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.054366    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-workload-socket\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.054529 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.054431    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-workload-certs\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.054529 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.054455    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-data\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.054688 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.054666    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istiod-ca-cert\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.056343 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.056321    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-envoy\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.056444 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.056422    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-podinfo\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.080344 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.080317    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-token\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.082923 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.082895    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm8nd\" (UniqueName: \"kubernetes.io/projected/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-kube-api-access-rm8nd\") pod \"e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") " pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.155308 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.155197    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:09.300236 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.300205    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"]
Jul 16 16:06:09.302860 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:06:09.302836    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41d47b2_77b2_4b38_aebb_a31d3b5a0911.slice/crio-d9558e35f336639df0f0b41b3fdb1963c367e30cfce91146f9899b2b23c7cee9 WatchSource:0}: Error finding container d9558e35f336639df0f0b41b3fdb1963c367e30cfce91146f9899b2b23c7cee9: Status 404 returned error can't find the container with id d9558e35f336639df0f0b41b3fdb1963c367e30cfce91146f9899b2b23c7cee9
Jul 16 16:06:09.304977 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.304951    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:06:09.305056 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.305013    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:06:09.305056 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.305045    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:06:09.635280 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.635240    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5" event={"ID":"d41d47b2-77b2-4b38-aebb-a31d3b5a0911","Type":"ContainerStarted","Data":"65267a79dfc1d08f257b397c62d3a10aa2508c3b8dd6f814b6c58f7a054529d0"}
Jul 16 16:06:09.635463 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.635308    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5" event={"ID":"d41d47b2-77b2-4b38-aebb-a31d3b5a0911","Type":"ContainerStarted","Data":"d9558e35f336639df0f0b41b3fdb1963c367e30cfce91146f9899b2b23c7cee9"}
Jul 16 16:06:09.658635 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:09.658584    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5" podStartSLOduration=1.658568232 podStartE2EDuration="1.658568232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:06:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:06:09.657718482 +0000 UTC m=+1835.846582438" watchObservedRunningTime="2026-07-16 16:06:09.658568232 +0000 UTC m=+1835.847432186"
Jul 16 16:06:10.156045 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:10.156006    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:10.161000 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:10.160976    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:10.643457 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:10.643425    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:10.644285 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:10.644262    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:11.243011 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:11.242974    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:11.620311 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:11.620280    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:11.705606 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:11.705576    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:12.031498 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:12.031414    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:12.290459 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:12.290375    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:12.387474 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:12.387444    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:12.487313 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:12.487280    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:16.159904 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.159870    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:16.529645 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.529555    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:16.815450 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.815369    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"]
Jul 16 16:06:16.815811 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.815793    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerName="istio-proxy"
Jul 16 16:06:16.815913 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.815813    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="dab76c4c-0e0a-4a36-bdac-b8463fe6fc61" containerName="istio-proxy"
Jul 16 16:06:16.819313 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.819288    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:16.822153 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.822129    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-7b7961-openshift-default-dockercfg-mmcmd\""
Jul 16 16:06:16.834494 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.834466    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"]
Jul 16 16:06:16.839653 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.839632    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:16.930388 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.930345    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-credential-socket\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:16.930583 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.930408    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-envoy\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:16.930583 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.930436    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-token\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:16.930583 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.930460    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istiod-ca-cert\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:16.930583 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.930487    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-data\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:16.930744 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.930592    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-workload-socket\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:16.930744 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.930635    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-workload-certs\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:16.930744 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.930681    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-podinfo\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:16.930744 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.930699    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87v4s\" (UniqueName: \"kubernetes.io/projected/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-kube-api-access-87v4s\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:16.943109 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:16.943079    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:17.031729 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.031694    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-workload-socket\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.031926 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.031745    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-workload-certs\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.031926 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.031789    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-podinfo\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.031926 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.031812    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87v4s\" (UniqueName: \"kubernetes.io/projected/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-kube-api-access-87v4s\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.031926 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.031844    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-credential-socket\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.031926 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.031891    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-envoy\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.031926 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.031916    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-token\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.032277 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.031973    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istiod-ca-cert\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.032277 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.032011    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-data\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.032277 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.032160    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-workload-socket\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.032277 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.032201    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-workload-certs\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.032277 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.032248    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-credential-socket\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.032503 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.032386    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-data\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.032702 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.032652    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istiod-ca-cert\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.034181 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.034156    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-envoy\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.034407 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.034392    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-podinfo\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.042668 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.042645    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87v4s\" (UniqueName: \"kubernetes.io/projected/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-kube-api-access-87v4s\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.042668 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.042659    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-token\") pod \"e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") " pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.130150 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.130114    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:17.274454 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.274427    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"]
Jul 16 16:06:17.276366 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:06:17.276339    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1605e2f8_ad1f_4fc3_adda_fadfaa260c9b.slice/crio-43c29443a1b28d8cd05b2bcc2b714ec6352481517ccc6f13408f7c8703d03b80 WatchSource:0}: Error finding container 43c29443a1b28d8cd05b2bcc2b714ec6352481517ccc6f13408f7c8703d03b80: Status 404 returned error can't find the container with id 43c29443a1b28d8cd05b2bcc2b714ec6352481517ccc6f13408f7c8703d03b80
Jul 16 16:06:17.278595 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.278562    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:06:17.278710 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.278647    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:06:17.278710 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.278693    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:06:17.675641 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.675604    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd" event={"ID":"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b","Type":"ContainerStarted","Data":"a12eae9eaeda466b676c40b11e3a709eb7608487205372334aad53ce08e82eef"}
Jul 16 16:06:17.675831 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.675666    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd" event={"ID":"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b","Type":"ContainerStarted","Data":"43c29443a1b28d8cd05b2bcc2b714ec6352481517ccc6f13408f7c8703d03b80"}
Jul 16 16:06:17.698789 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:17.698704    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd" podStartSLOduration=1.6986872549999998 podStartE2EDuration="1.698687255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:06:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:06:17.697264556 +0000 UTC m=+1843.886128512" watchObservedRunningTime="2026-07-16 16:06:17.698687255 +0000 UTC m=+1843.887551212"
Jul 16 16:06:18.131287 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:18.131244    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:18.136136 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:18.136110    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:18.680001 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:18.679970    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:18.681084 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:18.681063    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:19.661463 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:19.661427    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:21.129157 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:21.129124    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:21.188057 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:21.188025    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:24.186880 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:24.186842    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:26.685170 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:26.685127    2570 patch_prober.go:28] interesting pod/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 16 16:06:26.685564 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:26.685199    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5" podUID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 16 16:06:28.736290 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:28.736254    2570 generic.go:356] "Generic (PLEG): container finished" podID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" containerID="65267a79dfc1d08f257b397c62d3a10aa2508c3b8dd6f814b6c58f7a054529d0" exitCode=0
Jul 16 16:06:28.737039 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:28.736890    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5" event={"ID":"d41d47b2-77b2-4b38-aebb-a31d3b5a0911","Type":"ContainerDied","Data":"65267a79dfc1d08f257b397c62d3a10aa2508c3b8dd6f814b6c58f7a054529d0"}
Jul 16 16:06:28.737717 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:28.737699    2570 scope.go:117] "RemoveContainer" containerID="65267a79dfc1d08f257b397c62d3a10aa2508c3b8dd6f814b6c58f7a054529d0"
Jul 16 16:06:28.739747 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:28.739711    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:06:28.739970 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:28.739944    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:06:28.740135 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:28.740113    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:06:29.747092 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:29.747051    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5" event={"ID":"d41d47b2-77b2-4b38-aebb-a31d3b5a0911","Type":"ContainerStarted","Data":"b7f238b3d8b8c7217ca899098441d39e20b8de169a598bacab6f5648eacecc0d"}
Jul 16 16:06:30.156360 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:30.156303    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:30.156715 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:30.156382    2570 patch_prober.go:28] interesting pod/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 16 16:06:30.156906 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:30.156874    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5" podUID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 16 16:06:31.156520 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:31.156472    2570 patch_prober.go:28] interesting pod/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 16 16:06:31.157271 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:31.156545    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5" podUID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 16 16:06:32.156258 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:32.156221    2570 patch_prober.go:28] interesting pod/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 16 16:06:32.156449 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:32.156298    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5" podUID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 16 16:06:33.156221 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:33.156179    2570 patch_prober.go:28] interesting pod/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 16 16:06:33.156617 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:33.156252    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5" podUID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 16 16:06:33.284237 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:33.284207    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:33.386141 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:33.386112    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:34.159410 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:34.159369    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:34.159916 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:34.159756    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:34.160535 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:34.160514    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:06:34.555472 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:34.554200    2570 patch_prober.go:28] interesting pod/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": read tcp 10.133.0.2:45752->10.133.0.52:15021: read: connection reset by peer" start-of-body=
Jul 16 16:06:34.555472 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:34.554272    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd" podUID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": read tcp 10.133.0.2:45752->10.133.0.52:15021: read: connection reset by peer"
Jul 16 16:06:34.778691 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:34.778604    2570 generic.go:356] "Generic (PLEG): container finished" podID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" containerID="a12eae9eaeda466b676c40b11e3a709eb7608487205372334aad53ce08e82eef" exitCode=0
Jul 16 16:06:34.778691 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:34.778678    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd" event={"ID":"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b","Type":"ContainerDied","Data":"a12eae9eaeda466b676c40b11e3a709eb7608487205372334aad53ce08e82eef"}
Jul 16 16:06:34.779191 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:34.779172    2570 scope.go:117] "RemoveContainer" containerID="a12eae9eaeda466b676c40b11e3a709eb7608487205372334aad53ce08e82eef"
Jul 16 16:06:34.780087 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:34.780063    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:06:34.780154 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:34.780130    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:06:34.780196 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:34.780162    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:06:35.228717 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:35.228672    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:35.785015 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:35.784973    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd" event={"ID":"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b","Type":"ContainerStarted","Data":"e0c32e813120083e593e9a09d45f321ab49a8f30b775a90c138ceb592fcdadd6"}
Jul 16 16:06:36.131181 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:36.130449    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:36.131181 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:36.130846    2570 patch_prober.go:28] interesting pod/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 16 16:06:36.131181 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:36.130895    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd" podUID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 16 16:06:37.131150 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:37.131109    2570 patch_prober.go:28] interesting pod/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 16 16:06:37.131797 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:37.131762    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd" podUID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 16 16:06:38.130769 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:38.130728    2570 patch_prober.go:28] interesting pod/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 16 16:06:38.131109 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:38.131077    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd" podUID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 16 16:06:39.131157 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:39.131116    2570 patch_prober.go:28] interesting pod/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 16 16:06:39.131667 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:39.131185    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd" podUID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 16 16:06:40.134049 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:40.134019    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:40.134458 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:40.134350    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:40.135180 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:40.135161    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:06:42.491074 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:42.491038    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:06:42.595037 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:06:42.595002    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:07:17.327807 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.327724    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"]
Jul 16 16:07:17.331403 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.331385    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.334300 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.334259    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7b7961\"/\"kube-root-ca.crt\""
Jul 16 16:07:17.334565 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.334547    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7b7961\"/\"openshift-service-ca.crt\""
Jul 16 16:07:17.335481 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.335462    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7b7961\"/\"default-dockercfg-grzmq\""
Jul 16 16:07:17.335583 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.335511    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7b7961\"/\"test-model-7b7961-kserve-self-signed-certs\""
Jul 16 16:07:17.342911 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.342887    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"]
Jul 16 16:07:17.359327 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.359304    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:07:17.421296 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.421260    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-dshm\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.421473 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.421318    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-home\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.421473 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.421383    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-tls-certs\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.421473 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.421445    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-model-cache\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.421473 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.421468    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhdn5\" (UniqueName: \"kubernetes.io/projected/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-kube-api-access-rhdn5\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.421615 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.421543    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-tmp-dir\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.522104 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.522059    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-tmp-dir\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.522329 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.522126    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-dshm\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.522329 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.522187    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-home\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.522329 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.522214    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-tls-certs\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.522329 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.522245    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-model-cache\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.522329 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.522268    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhdn5\" (UniqueName: \"kubernetes.io/projected/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-kube-api-access-rhdn5\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.522643 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.522607    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-tmp-dir\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.522643 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.522634    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-model-cache\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.522794 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.522733    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-home\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.524541 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.524511    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-dshm\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.524765 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.524746    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-tls-certs\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.531278 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.531256    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhdn5\" (UniqueName: \"kubernetes.io/projected/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-kube-api-access-rhdn5\") pod \"test-model-7b7961-kserve-8687f4c7c7-5v5hc\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") " pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.642162 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.642130    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:17.778115 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.778089    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"]
Jul 16 16:07:17.780384 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:07:17.780357    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f16dcbd_afc7_406a_8f16_2e1fe1a84675.slice/crio-38c97bb826381da3a89c2786432e86a841ebaddf043a5aaef63af359d1789f30 WatchSource:0}: Error finding container 38c97bb826381da3a89c2786432e86a841ebaddf043a5aaef63af359d1789f30: Status 404 returned error can't find the container with id 38c97bb826381da3a89c2786432e86a841ebaddf043a5aaef63af359d1789f30
Jul 16 16:07:17.984922 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.984826    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc" event={"ID":"4f16dcbd-afc7-406a-8f16-2e1fe1a84675","Type":"ContainerStarted","Data":"464fb6b166e8513f7c3f55e4b5733fd93c03cfa3707d39e2819df45ae051b976"}
Jul 16 16:07:17.984922 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:17.984877    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc" event={"ID":"4f16dcbd-afc7-406a-8f16-2e1fe1a84675","Type":"ContainerStarted","Data":"38c97bb826381da3a89c2786432e86a841ebaddf043a5aaef63af359d1789f30"}
Jul 16 16:07:18.007178 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:18.007116    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc" podStartSLOduration=1.007100657 podStartE2EDuration="1.007100657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:07:18.00492735 +0000 UTC m=+1904.193791306" watchObservedRunningTime="2026-07-16 16:07:18.007100657 +0000 UTC m=+1904.195964612"
Jul 16 16:07:27.642731 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:27.642690    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:27.643148 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:27.642840    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:27.655874 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:27.655850    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:28.044621 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:28.044532    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:38.353824 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:38.353790    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:07:39.271289 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:39.271257    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"]
Jul 16 16:07:39.280200 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:39.280172    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:07:40.677758 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:40.677726    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:07:40.749150 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:40.749119    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:07:45.128003 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:45.127965    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:07:53.465542 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:53.465512    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:07:53.562725 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:53.562696    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:07:54.291448 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.291412    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc" podUID="4f16dcbd-afc7-406a-8f16-2e1fe1a84675" containerName="main" containerID="cri-o://464fb6b166e8513f7c3f55e4b5733fd93c03cfa3707d39e2819df45ae051b976" gracePeriod=45
Jul 16 16:07:54.554419 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.554354    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:54.672598 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.672566    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-home\") pod \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") "
Jul 16 16:07:54.672598 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.672605    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-tmp-dir\") pod \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") "
Jul 16 16:07:54.672872 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.672646    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-dshm\") pod \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") "
Jul 16 16:07:54.672872 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.672675    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-tls-certs\") pod \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") "
Jul 16 16:07:54.672872 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.672704    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhdn5\" (UniqueName: \"kubernetes.io/projected/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-kube-api-access-rhdn5\") pod \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") "
Jul 16 16:07:54.672872 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.672771    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-model-cache\") pod \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\" (UID: \"4f16dcbd-afc7-406a-8f16-2e1fe1a84675\") "
Jul 16 16:07:54.673098 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.672908    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-home" (OuterVolumeSpecName: "home") pod "4f16dcbd-afc7-406a-8f16-2e1fe1a84675" (UID: "4f16dcbd-afc7-406a-8f16-2e1fe1a84675"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:07:54.673098 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.672923    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4f16dcbd-afc7-406a-8f16-2e1fe1a84675" (UID: "4f16dcbd-afc7-406a-8f16-2e1fe1a84675"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:07:54.673207 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.673142    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-home\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:07:54.673207 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.673162    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-tmp-dir\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:07:54.673207 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.673162    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-model-cache" (OuterVolumeSpecName: "model-cache") pod "4f16dcbd-afc7-406a-8f16-2e1fe1a84675" (UID: "4f16dcbd-afc7-406a-8f16-2e1fe1a84675"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:07:54.675443 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.675389    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-dshm" (OuterVolumeSpecName: "dshm") pod "4f16dcbd-afc7-406a-8f16-2e1fe1a84675" (UID: "4f16dcbd-afc7-406a-8f16-2e1fe1a84675"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:07:54.675562 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.675461    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4f16dcbd-afc7-406a-8f16-2e1fe1a84675" (UID: "4f16dcbd-afc7-406a-8f16-2e1fe1a84675"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 16:07:54.675562 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.675523    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-kube-api-access-rhdn5" (OuterVolumeSpecName: "kube-api-access-rhdn5") pod "4f16dcbd-afc7-406a-8f16-2e1fe1a84675" (UID: "4f16dcbd-afc7-406a-8f16-2e1fe1a84675"). InnerVolumeSpecName "kube-api-access-rhdn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:07:54.774009 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.773970    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-dshm\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:07:54.774009 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.774003    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-tls-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:07:54.774009 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.774013    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhdn5\" (UniqueName: \"kubernetes.io/projected/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-kube-api-access-rhdn5\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:07:54.774009 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:54.774022    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f16dcbd-afc7-406a-8f16-2e1fe1a84675-model-cache\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:07:55.165358 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:55.165321    2570 generic.go:356] "Generic (PLEG): container finished" podID="4f16dcbd-afc7-406a-8f16-2e1fe1a84675" containerID="464fb6b166e8513f7c3f55e4b5733fd93c03cfa3707d39e2819df45ae051b976" exitCode=0
Jul 16 16:07:55.165568 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:55.165368    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"
Jul 16 16:07:55.165568 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:55.165407    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc" event={"ID":"4f16dcbd-afc7-406a-8f16-2e1fe1a84675","Type":"ContainerDied","Data":"464fb6b166e8513f7c3f55e4b5733fd93c03cfa3707d39e2819df45ae051b976"}
Jul 16 16:07:55.165568 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:55.165445    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc" event={"ID":"4f16dcbd-afc7-406a-8f16-2e1fe1a84675","Type":"ContainerDied","Data":"38c97bb826381da3a89c2786432e86a841ebaddf043a5aaef63af359d1789f30"}
Jul 16 16:07:55.165568 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:55.165463    2570 scope.go:117] "RemoveContainer" containerID="464fb6b166e8513f7c3f55e4b5733fd93c03cfa3707d39e2819df45ae051b976"
Jul 16 16:07:55.174974 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:55.174955    2570 scope.go:117] "RemoveContainer" containerID="464fb6b166e8513f7c3f55e4b5733fd93c03cfa3707d39e2819df45ae051b976"
Jul 16 16:07:55.175231 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:07:55.175211    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"464fb6b166e8513f7c3f55e4b5733fd93c03cfa3707d39e2819df45ae051b976\": container with ID starting with 464fb6b166e8513f7c3f55e4b5733fd93c03cfa3707d39e2819df45ae051b976 not found: ID does not exist" containerID="464fb6b166e8513f7c3f55e4b5733fd93c03cfa3707d39e2819df45ae051b976"
Jul 16 16:07:55.175280 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:55.175239    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"464fb6b166e8513f7c3f55e4b5733fd93c03cfa3707d39e2819df45ae051b976"} err="failed to get container status \"464fb6b166e8513f7c3f55e4b5733fd93c03cfa3707d39e2819df45ae051b976\": rpc error: code = NotFound desc = could not find container \"464fb6b166e8513f7c3f55e4b5733fd93c03cfa3707d39e2819df45ae051b976\": container with ID starting with 464fb6b166e8513f7c3f55e4b5733fd93c03cfa3707d39e2819df45ae051b976 not found: ID does not exist"
Jul 16 16:07:55.186449 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:55.186425    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"]
Jul 16 16:07:55.190389 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:55.190365    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-7b7961/test-model-7b7961-kserve-8687f4c7c7-5v5hc"]
Jul 16 16:07:56.267504 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:07:56.267459    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f16dcbd-afc7-406a-8f16-2e1fe1a84675" path="/var/lib/kubelet/pods/4f16dcbd-afc7-406a-8f16-2e1fe1a84675/volumes"
Jul 16 16:08:22.263116 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:22.263081    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:22.563436 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:22.563340    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"]
Jul 16 16:08:22.563671 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:22.563578    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5" podUID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" containerName="istio-proxy" containerID="cri-o://b7f238b3d8b8c7217ca899098441d39e20b8de169a598bacab6f5648eacecc0d" gracePeriod=30
Jul 16 16:08:22.585069 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:22.585040    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:22.836510 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:22.836465    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:23.023511 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:23.023476    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:23.454693 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:23.454659    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:23.552021 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:23.551990    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:23.653768 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:23.653736    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:23.777683 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:23.777590    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:27.838507 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.838484    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:08:27.985395 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.985360    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-envoy\") pod \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") "
Jul 16 16:08:27.985594 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.985410    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-credential-socket\") pod \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") "
Jul 16 16:08:27.985594 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.985433    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-workload-socket\") pod \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") "
Jul 16 16:08:27.985594 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.985474    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istiod-ca-cert\") pod \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") "
Jul 16 16:08:27.985594 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.985494    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-podinfo\") pod \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") "
Jul 16 16:08:27.985594 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.985515    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rm8nd\" (UniqueName: \"kubernetes.io/projected/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-kube-api-access-rm8nd\") pod \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") "
Jul 16 16:08:27.985594 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.985564    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-workload-certs\") pod \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") "
Jul 16 16:08:27.985910 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.985633    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-token\") pod \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") "
Jul 16 16:08:27.985910 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.985730    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d41d47b2-77b2-4b38-aebb-a31d3b5a0911" (UID: "d41d47b2-77b2-4b38-aebb-a31d3b5a0911"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:08:27.985910 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.985851    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d41d47b2-77b2-4b38-aebb-a31d3b5a0911" (UID: "d41d47b2-77b2-4b38-aebb-a31d3b5a0911"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:08:27.985910 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.985868    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d41d47b2-77b2-4b38-aebb-a31d3b5a0911" (UID: "d41d47b2-77b2-4b38-aebb-a31d3b5a0911"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:08:27.986173 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.985987    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-data\") pod \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\" (UID: \"d41d47b2-77b2-4b38-aebb-a31d3b5a0911\") "
Jul 16 16:08:27.986173 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.985984    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d41d47b2-77b2-4b38-aebb-a31d3b5a0911" (UID: "d41d47b2-77b2-4b38-aebb-a31d3b5a0911"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 16:08:27.986433 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.986408    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-workload-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:27.986528 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.986437    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-credential-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:27.986528 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.986452    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-workload-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:27.986528 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.986465    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istiod-ca-cert\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:27.987008 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.986983    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-data" (OuterVolumeSpecName: "istio-data") pod "d41d47b2-77b2-4b38-aebb-a31d3b5a0911" (UID: "d41d47b2-77b2-4b38-aebb-a31d3b5a0911"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:08:27.987751 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.987722    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d41d47b2-77b2-4b38-aebb-a31d3b5a0911" (UID: "d41d47b2-77b2-4b38-aebb-a31d3b5a0911"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:08:27.987751 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.987742    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d41d47b2-77b2-4b38-aebb-a31d3b5a0911" (UID: "d41d47b2-77b2-4b38-aebb-a31d3b5a0911"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 16:08:27.987869 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.987800    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-kube-api-access-rm8nd" (OuterVolumeSpecName: "kube-api-access-rm8nd") pod "d41d47b2-77b2-4b38-aebb-a31d3b5a0911" (UID: "d41d47b2-77b2-4b38-aebb-a31d3b5a0911"). InnerVolumeSpecName "kube-api-access-rm8nd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:08:27.987923 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:27.987910    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-token" (OuterVolumeSpecName: "istio-token") pod "d41d47b2-77b2-4b38-aebb-a31d3b5a0911" (UID: "d41d47b2-77b2-4b38-aebb-a31d3b5a0911"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:08:28.087072 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.087037    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-token\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:28.087072 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.087064    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-data\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:28.087072 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.087074    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-envoy\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:28.087072 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.087082    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-istio-podinfo\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:28.087343 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.087091    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rm8nd\" (UniqueName: \"kubernetes.io/projected/d41d47b2-77b2-4b38-aebb-a31d3b5a0911-kube-api-access-rm8nd\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:28.325157 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.325125    2570 generic.go:356] "Generic (PLEG): container finished" podID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" containerID="b7f238b3d8b8c7217ca899098441d39e20b8de169a598bacab6f5648eacecc0d" exitCode=0
Jul 16 16:08:28.325326 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.325171    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"
Jul 16 16:08:28.325326 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.325202    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5" event={"ID":"d41d47b2-77b2-4b38-aebb-a31d3b5a0911","Type":"ContainerDied","Data":"b7f238b3d8b8c7217ca899098441d39e20b8de169a598bacab6f5648eacecc0d"}
Jul 16 16:08:28.325326 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.325263    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5" event={"ID":"d41d47b2-77b2-4b38-aebb-a31d3b5a0911","Type":"ContainerDied","Data":"d9558e35f336639df0f0b41b3fdb1963c367e30cfce91146f9899b2b23c7cee9"}
Jul 16 16:08:28.325326 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.325294    2570 scope.go:117] "RemoveContainer" containerID="b7f238b3d8b8c7217ca899098441d39e20b8de169a598bacab6f5648eacecc0d"
Jul 16 16:08:28.334079 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.334057    2570 scope.go:117] "RemoveContainer" containerID="65267a79dfc1d08f257b397c62d3a10aa2508c3b8dd6f814b6c58f7a054529d0"
Jul 16 16:08:28.342049 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.342006    2570 scope.go:117] "RemoveContainer" containerID="b7f238b3d8b8c7217ca899098441d39e20b8de169a598bacab6f5648eacecc0d"
Jul 16 16:08:28.342267 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:08:28.342246    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7f238b3d8b8c7217ca899098441d39e20b8de169a598bacab6f5648eacecc0d\": container with ID starting with b7f238b3d8b8c7217ca899098441d39e20b8de169a598bacab6f5648eacecc0d not found: ID does not exist" containerID="b7f238b3d8b8c7217ca899098441d39e20b8de169a598bacab6f5648eacecc0d"
Jul 16 16:08:28.342311 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.342277    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7f238b3d8b8c7217ca899098441d39e20b8de169a598bacab6f5648eacecc0d"} err="failed to get container status \"b7f238b3d8b8c7217ca899098441d39e20b8de169a598bacab6f5648eacecc0d\": rpc error: code = NotFound desc = could not find container \"b7f238b3d8b8c7217ca899098441d39e20b8de169a598bacab6f5648eacecc0d\": container with ID starting with b7f238b3d8b8c7217ca899098441d39e20b8de169a598bacab6f5648eacecc0d not found: ID does not exist"
Jul 16 16:08:28.342311 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.342294    2570 scope.go:117] "RemoveContainer" containerID="65267a79dfc1d08f257b397c62d3a10aa2508c3b8dd6f814b6c58f7a054529d0"
Jul 16 16:08:28.342498 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:08:28.342483    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65267a79dfc1d08f257b397c62d3a10aa2508c3b8dd6f814b6c58f7a054529d0\": container with ID starting with 65267a79dfc1d08f257b397c62d3a10aa2508c3b8dd6f814b6c58f7a054529d0 not found: ID does not exist" containerID="65267a79dfc1d08f257b397c62d3a10aa2508c3b8dd6f814b6c58f7a054529d0"
Jul 16 16:08:28.342545 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.342502    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65267a79dfc1d08f257b397c62d3a10aa2508c3b8dd6f814b6c58f7a054529d0"} err="failed to get container status \"65267a79dfc1d08f257b397c62d3a10aa2508c3b8dd6f814b6c58f7a054529d0\": rpc error: code = NotFound desc = could not find container \"65267a79dfc1d08f257b397c62d3a10aa2508c3b8dd6f814b6c58f7a054529d0\": container with ID starting with 65267a79dfc1d08f257b397c62d3a10aa2508c3b8dd6f814b6c58f7a054529d0 not found: ID does not exist"
Jul 16 16:08:28.347765 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.347743    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"]
Jul 16 16:08:28.352687 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:28.352662    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-b8455d-openshift-default-66b98fdd9c-h5xk5"]
Jul 16 16:08:29.656166 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:29.656130    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:29.709463 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:29.709432    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:29.922457 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:29.922373    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"]
Jul 16 16:08:29.922648 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:29.922608    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd" podUID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" containerName="istio-proxy" containerID="cri-o://e0c32e813120083e593e9a09d45f321ab49a8f30b775a90c138ceb592fcdadd6" gracePeriod=30
Jul 16 16:08:29.939044 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:29.939013    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:30.188402 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.188314    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:30.267361 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.267332    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" path="/var/lib/kubelet/pods/d41d47b2-77b2-4b38-aebb-a31d3b5a0911/volumes"
Jul 16 16:08:30.528485 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.528396    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:30.854314 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.854282    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"]
Jul 16 16:08:30.854694 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.854649    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" containerName="istio-proxy"
Jul 16 16:08:30.854694 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.854661    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" containerName="istio-proxy"
Jul 16 16:08:30.854694 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.854676    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" containerName="istio-proxy"
Jul 16 16:08:30.854694 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.854682    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" containerName="istio-proxy"
Jul 16 16:08:30.854694 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.854688    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f16dcbd-afc7-406a-8f16-2e1fe1a84675" containerName="main"
Jul 16 16:08:30.854694 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.854693    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f16dcbd-afc7-406a-8f16-2e1fe1a84675" containerName="main"
Jul 16 16:08:30.854950 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.854753    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" containerName="istio-proxy"
Jul 16 16:08:30.854950 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.854760    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f16dcbd-afc7-406a-8f16-2e1fe1a84675" containerName="main"
Jul 16 16:08:30.857977 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.857958    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:30.861048 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.861023    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-bb0c28-openshift-default-dockercfg-6d89m\""
Jul 16 16:08:30.874381 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.874354    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"]
Jul 16 16:08:30.874624 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.874600    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:30.910690 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.910643    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-workload-certs\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:30.910690 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.910690    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-envoy\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:30.910988 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.910761    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-data\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:30.910988 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.910793    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-workload-socket\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:30.910988 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.910813    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-credential-socket\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:30.910988 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.910844    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a12ba9a8-4295-45b0-bf25-494a855300c9-istiod-ca-cert\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:30.910988 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.910881    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7gst\" (UniqueName: \"kubernetes.io/projected/a12ba9a8-4295-45b0-bf25-494a855300c9-kube-api-access-l7gst\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:30.910988 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.910918    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-token\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:30.910988 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:30.910989    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-podinfo\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.012097 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.012064    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-podinfo\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.012294 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.012126    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-workload-certs\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.012294 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.012144    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-envoy\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.012294 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.012178    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-data\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.012294 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.012203    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-workload-socket\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.012294 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.012220    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-credential-socket\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.012294 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.012262    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a12ba9a8-4295-45b0-bf25-494a855300c9-istiod-ca-cert\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.012294 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.012290    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7gst\" (UniqueName: \"kubernetes.io/projected/a12ba9a8-4295-45b0-bf25-494a855300c9-kube-api-access-l7gst\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.012686 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.012324    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-token\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.012742 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.012675    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-workload-certs\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.012742 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.012705    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-credential-socket\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.013134 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.013104    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a12ba9a8-4295-45b0-bf25-494a855300c9-istiod-ca-cert\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.013270 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.013104    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-workload-socket\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.013270 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.013156    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-data\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.014660 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.014635    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-envoy\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.014775 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.014759    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-podinfo\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.025639 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.025613    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-token\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.025769 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.025693    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7gst\" (UniqueName: \"kubernetes.io/projected/a12ba9a8-4295-45b0-bf25-494a855300c9-kube-api-access-l7gst\") pod \"e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") " pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.041128 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.041099    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:31.169388 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.169287    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:31.324977 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.324803    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"]
Jul 16 16:08:31.327790 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:08:31.327759    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda12ba9a8_4295_45b0_bf25_494a855300c9.slice/crio-aeb62f622ecc468ec1a93556900b3c49baf16190a37d672acd67d4a70dc833ff WatchSource:0}: Error finding container aeb62f622ecc468ec1a93556900b3c49baf16190a37d672acd67d4a70dc833ff: Status 404 returned error can't find the container with id aeb62f622ecc468ec1a93556900b3c49baf16190a37d672acd67d4a70dc833ff
Jul 16 16:08:31.329913 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.329885    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:08:31.330021 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.329967    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:08:31.330021 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.330000    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:08:31.346993 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:31.346960    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd" event={"ID":"a12ba9a8-4295-45b0-bf25-494a855300c9","Type":"ContainerStarted","Data":"aeb62f622ecc468ec1a93556900b3c49baf16190a37d672acd67d4a70dc833ff"}
Jul 16 16:08:32.353564 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:32.353528    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd" event={"ID":"a12ba9a8-4295-45b0-bf25-494a855300c9","Type":"ContainerStarted","Data":"777f0feaa852d9210b407774ca24ac0ed324a7736e22794de86cd8e9f4a9768b"}
Jul 16 16:08:32.379520 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:32.379468    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd" podStartSLOduration=2.379450557 podStartE2EDuration="2.379450557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:08:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:08:32.377395198 +0000 UTC m=+1978.566259168" watchObservedRunningTime="2026-07-16 16:08:32.379450557 +0000 UTC m=+1978.568314572"
Jul 16 16:08:33.170091 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:33.170057    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:33.175138 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:33.175113    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:33.358623 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:33.358597    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:33.359522 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:33.359501    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:34.543870 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:34.543838    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:34.657360 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:34.657284    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:34.760782 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:34.760750    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:35.078422 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.078389    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:35.188508 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.188443    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:08:35.248665 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.248637    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-credential-socket\") pod \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") "
Jul 16 16:08:35.248852 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.248675    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-token\") pod \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") "
Jul 16 16:08:35.248852 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.248715    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-workload-certs\") pod \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") "
Jul 16 16:08:35.248852 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.248738    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-envoy\") pod \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") "
Jul 16 16:08:35.248852 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.248810    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-podinfo\") pod \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") "
Jul 16 16:08:35.248852 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.248850    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87v4s\" (UniqueName: \"kubernetes.io/projected/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-kube-api-access-87v4s\") pod \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") "
Jul 16 16:08:35.249137 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.248910    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-data\") pod \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") "
Jul 16 16:08:35.249137 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.248950    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-workload-socket\") pod \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") "
Jul 16 16:08:35.249137 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.248984    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istiod-ca-cert\") pod \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\" (UID: \"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b\") "
Jul 16 16:08:35.249137 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.248989    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" (UID: "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:08:35.249137 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.249114    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" (UID: "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:08:35.249377 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.249364    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-credential-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:35.249432 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.249386    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-workload-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:35.249432 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.249407    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" (UID: "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:08:35.249518 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.249435    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" (UID: "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 16:08:35.249764 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.249741    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-data" (OuterVolumeSpecName: "istio-data") pod "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" (UID: "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:08:35.251081 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.251058    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" (UID: "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 16:08:35.251215 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.251198    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-kube-api-access-87v4s" (OuterVolumeSpecName: "kube-api-access-87v4s") pod "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" (UID: "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b"). InnerVolumeSpecName "kube-api-access-87v4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:08:35.251447 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.251426    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" (UID: "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:08:35.251447 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.251435    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-token" (OuterVolumeSpecName: "istio-token") pod "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" (UID: "1605e2f8-ad1f-4fc3-adda-fadfaa260c9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:08:35.350131 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.350097    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-podinfo\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:35.350131 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.350122    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87v4s\" (UniqueName: \"kubernetes.io/projected/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-kube-api-access-87v4s\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:35.350131 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.350132    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-data\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:35.350131 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.350141    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-workload-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:35.350415 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.350151    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istiod-ca-cert\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:35.350415 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.350160    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-token\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:35.350415 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.350170    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b-istio-envoy\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:08:35.368680 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.368640    2570 generic.go:356] "Generic (PLEG): container finished" podID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" containerID="e0c32e813120083e593e9a09d45f321ab49a8f30b775a90c138ceb592fcdadd6" exitCode=0
Jul 16 16:08:35.368850 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.368702    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"
Jul 16 16:08:35.368850 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.368722    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd" event={"ID":"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b","Type":"ContainerDied","Data":"e0c32e813120083e593e9a09d45f321ab49a8f30b775a90c138ceb592fcdadd6"}
Jul 16 16:08:35.368850 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.368776    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd" event={"ID":"1605e2f8-ad1f-4fc3-adda-fadfaa260c9b","Type":"ContainerDied","Data":"43c29443a1b28d8cd05b2bcc2b714ec6352481517ccc6f13408f7c8703d03b80"}
Jul 16 16:08:35.368850 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.368800    2570 scope.go:117] "RemoveContainer" containerID="e0c32e813120083e593e9a09d45f321ab49a8f30b775a90c138ceb592fcdadd6"
Jul 16 16:08:35.378469 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.378450    2570 scope.go:117] "RemoveContainer" containerID="a12eae9eaeda466b676c40b11e3a709eb7608487205372334aad53ce08e82eef"
Jul 16 16:08:35.386792 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.386775    2570 scope.go:117] "RemoveContainer" containerID="e0c32e813120083e593e9a09d45f321ab49a8f30b775a90c138ceb592fcdadd6"
Jul 16 16:08:35.387147 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:08:35.387126    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0c32e813120083e593e9a09d45f321ab49a8f30b775a90c138ceb592fcdadd6\": container with ID starting with e0c32e813120083e593e9a09d45f321ab49a8f30b775a90c138ceb592fcdadd6 not found: ID does not exist" containerID="e0c32e813120083e593e9a09d45f321ab49a8f30b775a90c138ceb592fcdadd6"
Jul 16 16:08:35.387194 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.387157    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0c32e813120083e593e9a09d45f321ab49a8f30b775a90c138ceb592fcdadd6"} err="failed to get container status \"e0c32e813120083e593e9a09d45f321ab49a8f30b775a90c138ceb592fcdadd6\": rpc error: code = NotFound desc = could not find container \"e0c32e813120083e593e9a09d45f321ab49a8f30b775a90c138ceb592fcdadd6\": container with ID starting with e0c32e813120083e593e9a09d45f321ab49a8f30b775a90c138ceb592fcdadd6 not found: ID does not exist"
Jul 16 16:08:35.387194 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.387174    2570 scope.go:117] "RemoveContainer" containerID="a12eae9eaeda466b676c40b11e3a709eb7608487205372334aad53ce08e82eef"
Jul 16 16:08:35.387436 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:08:35.387421    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a12eae9eaeda466b676c40b11e3a709eb7608487205372334aad53ce08e82eef\": container with ID starting with a12eae9eaeda466b676c40b11e3a709eb7608487205372334aad53ce08e82eef not found: ID does not exist" containerID="a12eae9eaeda466b676c40b11e3a709eb7608487205372334aad53ce08e82eef"
Jul 16 16:08:35.387486 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.387441    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a12eae9eaeda466b676c40b11e3a709eb7608487205372334aad53ce08e82eef"} err="failed to get container status \"a12eae9eaeda466b676c40b11e3a709eb7608487205372334aad53ce08e82eef\": rpc error: code = NotFound desc = could not find container \"a12eae9eaeda466b676c40b11e3a709eb7608487205372334aad53ce08e82eef\": container with ID starting with a12eae9eaeda466b676c40b11e3a709eb7608487205372334aad53ce08e82eef not found: ID does not exist"
Jul 16 16:08:35.390969 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.390945    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:35.400387 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.400364    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"]
Jul 16 16:08:35.405990 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.405970    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-7b7961-openshift-default-7c8bdd75c9-q9wjd"]
Jul 16 16:08:35.493965 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:35.493853    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:36.267973 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:36.267919    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" path="/var/lib/kubelet/pods/1605e2f8-ad1f-4fc3-adda-fadfaa260c9b/volumes"
Jul 16 16:08:38.899412 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:38.899382    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:39.382039 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.382004    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:39.725361 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.725259    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"]
Jul 16 16:08:39.725865 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.725846    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" containerName="istio-proxy"
Jul 16 16:08:39.725865 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.725865    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" containerName="istio-proxy"
Jul 16 16:08:39.726051 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.725877    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" containerName="istio-proxy"
Jul 16 16:08:39.726051 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.725886    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" containerName="istio-proxy"
Jul 16 16:08:39.726051 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.725974    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" containerName="istio-proxy"
Jul 16 16:08:39.726051 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.725988    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d41d47b2-77b2-4b38-aebb-a31d3b5a0911" containerName="istio-proxy"
Jul 16 16:08:39.729261 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.729239    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.732298 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.732280    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-22d3cd-openshift-default-dockercfg-8xtgt\""
Jul 16 16:08:39.740249 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.740225    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"]
Jul 16 16:08:39.755197 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.755168    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:39.793352 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.793320    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-token\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.793352 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.793357    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e838476-d512-4410-91e4-3ad2b7fdf56b-istiod-ca-cert\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.793575 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.793383    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7xl5\" (UniqueName: \"kubernetes.io/projected/7e838476-d512-4410-91e4-3ad2b7fdf56b-kube-api-access-t7xl5\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.793575 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.793454    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-podinfo\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.793575 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.793512    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-workload-certs\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.793768 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.793592    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-workload-socket\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.793768 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.793659    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-data\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.793768 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.793690    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-credential-socket\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.793768 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.793716    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-envoy\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.895019 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.894978    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-podinfo\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.895019 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.895027    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-workload-certs\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.895257 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.895070    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-workload-socket\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.895257 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.895107    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-data\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.895257 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.895126    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-credential-socket\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.895257 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.895142    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-envoy\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.895257 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.895211    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-token\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.895257 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.895241    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e838476-d512-4410-91e4-3ad2b7fdf56b-istiod-ca-cert\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.895584 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.895266    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7xl5\" (UniqueName: \"kubernetes.io/projected/7e838476-d512-4410-91e4-3ad2b7fdf56b-kube-api-access-t7xl5\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.895584 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.895562    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-workload-certs\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.895680 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.895586    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-data\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.895750 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.895725    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-credential-socket\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.895811 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.895754    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-workload-socket\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.896215 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.896191    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e838476-d512-4410-91e4-3ad2b7fdf56b-istiod-ca-cert\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.897603 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.897576    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-envoy\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.897740 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.897720    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-podinfo\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.905366 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.905344    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-token\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:39.905727 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:39.905522    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7xl5\" (UniqueName: \"kubernetes.io/projected/7e838476-d512-4410-91e4-3ad2b7fdf56b-kube-api-access-t7xl5\") pod \"e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") " pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:40.042640 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:40.042544    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:40.068585 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:40.068554    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:40.188779 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:40.188668    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"]
Jul 16 16:08:40.191359 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:08:40.191327    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e838476_d512_4410_91e4_3ad2b7fdf56b.slice/crio-e78976dfbc4c17756a044fb54e5df75faf02ab64a0cac515cae7bdf5fa592219 WatchSource:0}: Error finding container e78976dfbc4c17756a044fb54e5df75faf02ab64a0cac515cae7bdf5fa592219: Status 404 returned error can't find the container with id e78976dfbc4c17756a044fb54e5df75faf02ab64a0cac515cae7bdf5fa592219
Jul 16 16:08:40.194153 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:40.194122    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:08:40.194248 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:40.194190    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:08:40.194248 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:40.194220    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:08:40.395165 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:40.395122    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n" event={"ID":"7e838476-d512-4410-91e4-3ad2b7fdf56b","Type":"ContainerStarted","Data":"724035b92a1f982ea7f96e0f4cd5878c8b3e71772ceba37a5258a61f7c740585"}
Jul 16 16:08:40.395346 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:40.395183    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n" event={"ID":"7e838476-d512-4410-91e4-3ad2b7fdf56b","Type":"ContainerStarted","Data":"e78976dfbc4c17756a044fb54e5df75faf02ab64a0cac515cae7bdf5fa592219"}
Jul 16 16:08:40.454589 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:40.454537    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n" podStartSLOduration=1.454516022 podStartE2EDuration="1.454516022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:08:40.444679873 +0000 UTC m=+1986.633543827" watchObservedRunningTime="2026-07-16 16:08:40.454516022 +0000 UTC m=+1986.643379977"
Jul 16 16:08:41.045023 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:41.044984    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:41.053652 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:41.053613    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:41.399986 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:41.399956    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:41.401077 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:41.401064    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:08:42.638888 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:42.638841    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:43.538171 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:43.538138    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:44.508593 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:44.508553    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:47.118290 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:47.118258    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:48.360015 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:48.359977    2570 patch_prober.go:28] interesting pod/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 16 16:08:48.360443 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:48.360032    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd" podUID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 16 16:08:48.433365 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:48.433334    2570 generic.go:356] "Generic (PLEG): container finished" podID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerID="777f0feaa852d9210b407774ca24ac0ed324a7736e22794de86cd8e9f4a9768b" exitCode=0
Jul 16 16:08:48.433532 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:48.433404    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd" event={"ID":"a12ba9a8-4295-45b0-bf25-494a855300c9","Type":"ContainerDied","Data":"777f0feaa852d9210b407774ca24ac0ed324a7736e22794de86cd8e9f4a9768b"}
Jul 16 16:08:48.433809 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:48.433795    2570 scope.go:117] "RemoveContainer" containerID="777f0feaa852d9210b407774ca24ac0ed324a7736e22794de86cd8e9f4a9768b"
Jul 16 16:08:48.434904 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:48.434874    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:08:48.434985 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:48.434965    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:08:48.435028 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:48.435020    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:08:49.440468 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:49.440433    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd" event={"ID":"a12ba9a8-4295-45b0-bf25-494a855300c9","Type":"ContainerStarted","Data":"a12c1165bb4780e31a3d5c65c91450171eb66eed9add9b2ac8639c525ffb53f7"}
Jul 16 16:08:50.169524 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:50.169489    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:50.169741 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:50.169688    2570 patch_prober.go:28] interesting pod/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 16 16:08:50.169818 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:50.169744    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd" podUID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 16 16:08:51.170347 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:51.170307    2570 patch_prober.go:28] interesting pod/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 16 16:08:51.170752 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:51.170384    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd" podUID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 16 16:08:52.170453 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:52.170405    2570 patch_prober.go:28] interesting pod/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 16 16:08:52.171009 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:52.170494    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd" podUID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 16 16:08:53.169812 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:53.169770    2570 patch_prober.go:28] interesting pod/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 16 16:08:53.170171 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:53.170136    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd" podUID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 16 16:08:54.170206 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:54.170155    2570 patch_prober.go:28] interesting pod/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 16 16:08:54.170773 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:54.170230    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd" podUID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 16 16:08:55.174589 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:55.174560    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:55.174995 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:55.174915    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:55.175687 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:55.175669    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:08:55.559254 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:55.559224    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:55.984024 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:55.983985    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:56.108123 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:56.108093    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:56.208670 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:56.208636    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:56.436223 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:56.436187    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:08:57.441101 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:57.441064    2570 patch_prober.go:28] interesting pod/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 16 16:08:57.441486 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:57.441139    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n" podUID="7e838476-d512-4410-91e4-3ad2b7fdf56b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 16 16:08:58.500576 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:58.500541    2570 generic.go:356] "Generic (PLEG): container finished" podID="7e838476-d512-4410-91e4-3ad2b7fdf56b" containerID="724035b92a1f982ea7f96e0f4cd5878c8b3e71772ceba37a5258a61f7c740585" exitCode=0
Jul 16 16:08:58.501060 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:58.500617    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n" event={"ID":"7e838476-d512-4410-91e4-3ad2b7fdf56b","Type":"ContainerDied","Data":"724035b92a1f982ea7f96e0f4cd5878c8b3e71772ceba37a5258a61f7c740585"}
Jul 16 16:08:58.501060 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:58.501025    2570 scope.go:117] "RemoveContainer" containerID="724035b92a1f982ea7f96e0f4cd5878c8b3e71772ceba37a5258a61f7c740585"
Jul 16 16:08:58.502060 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:58.502033    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:08:58.502136 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:58.502119    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:08:58.502177 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:58.502165    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:08:59.508005 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:08:59.507973    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n" event={"ID":"7e838476-d512-4410-91e4-3ad2b7fdf56b","Type":"ContainerStarted","Data":"8ee3622f634fbf1a7a4cd49672ccdf93d666f7c2d80496e97a2b55b0d3b939fd"}
Jul 16 16:09:00.043777 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:00.043684    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:09:00.044124 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:00.044097    2570 patch_prober.go:28] interesting pod/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 16 16:09:00.044312 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:00.044286    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n" podUID="7e838476-d512-4410-91e4-3ad2b7fdf56b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 16 16:09:01.043542 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:01.043502    2570 patch_prober.go:28] interesting pod/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 16 16:09:01.044189 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:01.044154    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n" podUID="7e838476-d512-4410-91e4-3ad2b7fdf56b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 16 16:09:02.043793 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:02.043746    2570 patch_prober.go:28] interesting pod/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 16 16:09:02.044314 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:02.043817    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n" podUID="7e838476-d512-4410-91e4-3ad2b7fdf56b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 16 16:09:02.753344 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:02.753308    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:02.855983 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:02.855922    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:02.969254 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:02.969222    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"]
Jul 16 16:09:02.970040 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:02.970007    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd" podUID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerName="istio-proxy" containerID="cri-o://a12c1165bb4780e31a3d5c65c91450171eb66eed9add9b2ac8639c525ffb53f7" gracePeriod=30
Jul 16 16:09:02.984892 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:02.984865    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:03.046171 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:03.046049    2570 patch_prober.go:28] interesting pod/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 16 16:09:03.046171 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:03.046134    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n" podUID="7e838476-d512-4410-91e4-3ad2b7fdf56b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 16 16:09:03.210070 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:03.210033    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:03.410214 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:03.410179    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:03.845989 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:03.845961    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:04.046602 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:04.046574    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:09:04.047012 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:04.046961    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:09:04.047536 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:04.047517    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:09:05.628412 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:05.628384    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:05.684861 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:05.684834    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:08.245514 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.245454    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:09:08.377252 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.377218    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-token\") pod \"a12ba9a8-4295-45b0-bf25-494a855300c9\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") "
Jul 16 16:09:08.377252 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.377256    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7gst\" (UniqueName: \"kubernetes.io/projected/a12ba9a8-4295-45b0-bf25-494a855300c9-kube-api-access-l7gst\") pod \"a12ba9a8-4295-45b0-bf25-494a855300c9\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") "
Jul 16 16:09:08.377510 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.377303    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-workload-socket\") pod \"a12ba9a8-4295-45b0-bf25-494a855300c9\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") "
Jul 16 16:09:08.377510 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.377327    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-workload-certs\") pod \"a12ba9a8-4295-45b0-bf25-494a855300c9\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") "
Jul 16 16:09:08.377510 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.377348    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-credential-socket\") pod \"a12ba9a8-4295-45b0-bf25-494a855300c9\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") "
Jul 16 16:09:08.377510 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.377380    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-envoy\") pod \"a12ba9a8-4295-45b0-bf25-494a855300c9\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") "
Jul 16 16:09:08.377510 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.377399    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a12ba9a8-4295-45b0-bf25-494a855300c9-istiod-ca-cert\") pod \"a12ba9a8-4295-45b0-bf25-494a855300c9\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") "
Jul 16 16:09:08.377510 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.377435    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-podinfo\") pod \"a12ba9a8-4295-45b0-bf25-494a855300c9\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") "
Jul 16 16:09:08.377510 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.377466    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-data\") pod \"a12ba9a8-4295-45b0-bf25-494a855300c9\" (UID: \"a12ba9a8-4295-45b0-bf25-494a855300c9\") "
Jul 16 16:09:08.377859 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.377664    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a12ba9a8-4295-45b0-bf25-494a855300c9" (UID: "a12ba9a8-4295-45b0-bf25-494a855300c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:09:08.377859 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.377674    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a12ba9a8-4295-45b0-bf25-494a855300c9" (UID: "a12ba9a8-4295-45b0-bf25-494a855300c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:09:08.377987 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.377877    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-workload-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:08.377987 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.377896    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-workload-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:08.377987 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.377914    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a12ba9a8-4295-45b0-bf25-494a855300c9" (UID: "a12ba9a8-4295-45b0-bf25-494a855300c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:09:08.378280 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.378255    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a12ba9a8-4295-45b0-bf25-494a855300c9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a12ba9a8-4295-45b0-bf25-494a855300c9" (UID: "a12ba9a8-4295-45b0-bf25-494a855300c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 16:09:08.378569 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.378548    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-data" (OuterVolumeSpecName: "istio-data") pod "a12ba9a8-4295-45b0-bf25-494a855300c9" (UID: "a12ba9a8-4295-45b0-bf25-494a855300c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:09:08.379696 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.379669    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a12ba9a8-4295-45b0-bf25-494a855300c9" (UID: "a12ba9a8-4295-45b0-bf25-494a855300c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:09:08.379818 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.379725    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-token" (OuterVolumeSpecName: "istio-token") pod "a12ba9a8-4295-45b0-bf25-494a855300c9" (UID: "a12ba9a8-4295-45b0-bf25-494a855300c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:09:08.380017 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.379985    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a12ba9a8-4295-45b0-bf25-494a855300c9" (UID: "a12ba9a8-4295-45b0-bf25-494a855300c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 16:09:08.380248 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.380231    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a12ba9a8-4295-45b0-bf25-494a855300c9-kube-api-access-l7gst" (OuterVolumeSpecName: "kube-api-access-l7gst") pod "a12ba9a8-4295-45b0-bf25-494a855300c9" (UID: "a12ba9a8-4295-45b0-bf25-494a855300c9"). InnerVolumeSpecName "kube-api-access-l7gst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:09:08.478897 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.478860    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-token\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:08.478897 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.478892    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7gst\" (UniqueName: \"kubernetes.io/projected/a12ba9a8-4295-45b0-bf25-494a855300c9-kube-api-access-l7gst\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:08.478897 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.478902    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-credential-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:08.479173 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.478911    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-envoy\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:08.479173 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.478922    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a12ba9a8-4295-45b0-bf25-494a855300c9-istiod-ca-cert\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:08.479173 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.478962    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-podinfo\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:08.479173 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.478974    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a12ba9a8-4295-45b0-bf25-494a855300c9-istio-data\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:08.551720 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.551622    2570 generic.go:356] "Generic (PLEG): container finished" podID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerID="a12c1165bb4780e31a3d5c65c91450171eb66eed9add9b2ac8639c525ffb53f7" exitCode=0
Jul 16 16:09:08.551720 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.551682    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"
Jul 16 16:09:08.551720 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.551706    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd" event={"ID":"a12ba9a8-4295-45b0-bf25-494a855300c9","Type":"ContainerDied","Data":"a12c1165bb4780e31a3d5c65c91450171eb66eed9add9b2ac8639c525ffb53f7"}
Jul 16 16:09:08.551976 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.551788    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd" event={"ID":"a12ba9a8-4295-45b0-bf25-494a855300c9","Type":"ContainerDied","Data":"aeb62f622ecc468ec1a93556900b3c49baf16190a37d672acd67d4a70dc833ff"}
Jul 16 16:09:08.551976 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.551811    2570 scope.go:117] "RemoveContainer" containerID="a12c1165bb4780e31a3d5c65c91450171eb66eed9add9b2ac8639c525ffb53f7"
Jul 16 16:09:08.561800 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.561784    2570 scope.go:117] "RemoveContainer" containerID="777f0feaa852d9210b407774ca24ac0ed324a7736e22794de86cd8e9f4a9768b"
Jul 16 16:09:08.570742 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.570716    2570 scope.go:117] "RemoveContainer" containerID="a12c1165bb4780e31a3d5c65c91450171eb66eed9add9b2ac8639c525ffb53f7"
Jul 16 16:09:08.571053 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:09:08.571029    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a12c1165bb4780e31a3d5c65c91450171eb66eed9add9b2ac8639c525ffb53f7\": container with ID starting with a12c1165bb4780e31a3d5c65c91450171eb66eed9add9b2ac8639c525ffb53f7 not found: ID does not exist" containerID="a12c1165bb4780e31a3d5c65c91450171eb66eed9add9b2ac8639c525ffb53f7"
Jul 16 16:09:08.571134 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.571066    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a12c1165bb4780e31a3d5c65c91450171eb66eed9add9b2ac8639c525ffb53f7"} err="failed to get container status \"a12c1165bb4780e31a3d5c65c91450171eb66eed9add9b2ac8639c525ffb53f7\": rpc error: code = NotFound desc = could not find container \"a12c1165bb4780e31a3d5c65c91450171eb66eed9add9b2ac8639c525ffb53f7\": container with ID starting with a12c1165bb4780e31a3d5c65c91450171eb66eed9add9b2ac8639c525ffb53f7 not found: ID does not exist"
Jul 16 16:09:08.571134 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.571090    2570 scope.go:117] "RemoveContainer" containerID="777f0feaa852d9210b407774ca24ac0ed324a7736e22794de86cd8e9f4a9768b"
Jul 16 16:09:08.571352 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:09:08.571317    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"777f0feaa852d9210b407774ca24ac0ed324a7736e22794de86cd8e9f4a9768b\": container with ID starting with 777f0feaa852d9210b407774ca24ac0ed324a7736e22794de86cd8e9f4a9768b not found: ID does not exist" containerID="777f0feaa852d9210b407774ca24ac0ed324a7736e22794de86cd8e9f4a9768b"
Jul 16 16:09:08.571393 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.571357    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"777f0feaa852d9210b407774ca24ac0ed324a7736e22794de86cd8e9f4a9768b"} err="failed to get container status \"777f0feaa852d9210b407774ca24ac0ed324a7736e22794de86cd8e9f4a9768b\": rpc error: code = NotFound desc = could not find container \"777f0feaa852d9210b407774ca24ac0ed324a7736e22794de86cd8e9f4a9768b\": container with ID starting with 777f0feaa852d9210b407774ca24ac0ed324a7736e22794de86cd8e9f4a9768b not found: ID does not exist"
Jul 16 16:09:08.577389 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.577362    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"]
Jul 16 16:09:08.582974 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:08.582925    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-bb0c28-openshift-default-54db4575b5-klljd"]
Jul 16 16:09:10.267365 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:10.267331    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a12ba9a8-4295-45b0-bf25-494a855300c9" path="/var/lib/kubelet/pods/a12ba9a8-4295-45b0-bf25-494a855300c9/volumes"
Jul 16 16:09:10.656373 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:10.656338    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:10.725539 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:10.725502    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:10.987764 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:10.987672    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"]
Jul 16 16:09:10.987916 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:10.987870    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n" podUID="7e838476-d512-4410-91e4-3ad2b7fdf56b" containerName="istio-proxy" containerID="cri-o://8ee3622f634fbf1a7a4cd49672ccdf93d666f7c2d80496e97a2b55b0d3b939fd" gracePeriod=30
Jul 16 16:09:11.007742 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:11.007709    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:11.227706 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:11.227677    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:11.583420 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:11.583384    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:11.996778 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:11.996747    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"]
Jul 16 16:09:11.997149 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:11.997136    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerName="istio-proxy"
Jul 16 16:09:11.997149 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:11.997150    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerName="istio-proxy"
Jul 16 16:09:11.997248 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:11.997160    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerName="istio-proxy"
Jul 16 16:09:11.997248 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:11.997167    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerName="istio-proxy"
Jul 16 16:09:11.997248 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:11.997226    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1605e2f8-ad1f-4fc3-adda-fadfaa260c9b" containerName="istio-proxy"
Jul 16 16:09:11.997248 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:11.997235    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerName="istio-proxy"
Jul 16 16:09:12.001784 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.001766    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.005652 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.005629    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-83e7d4-openshift-default-dockercfg-pgqxt\""
Jul 16 16:09:12.017404 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.017375    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:12.018980 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.018951    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"]
Jul 16 16:09:12.112460 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.112426    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istiod-ca-cert\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.112628 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.112477    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-credential-socket\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.112628 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.112512    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-envoy\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.112628 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.112593    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4shxw\" (UniqueName: \"kubernetes.io/projected/cc67b0d3-1f45-4971-81ad-214aa8c1b844-kube-api-access-4shxw\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.112756 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.112652    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-data\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.112756 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.112711    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-workload-certs\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.112756 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.112738    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-podinfo\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.112870 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.112778    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-workload-socket\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.112870 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.112805    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-token\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.213715 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.213676    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istiod-ca-cert\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.213919 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.213727    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-credential-socket\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.213919 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.213749    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-envoy\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.213919 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.213769    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4shxw\" (UniqueName: \"kubernetes.io/projected/cc67b0d3-1f45-4971-81ad-214aa8c1b844-kube-api-access-4shxw\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.213919 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.213802    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-data\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.213919 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.213853    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-workload-certs\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.213919 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.213883    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-podinfo\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.213919 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.213919    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-workload-socket\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.214328 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.213970    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-token\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.214328 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.214215    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-data\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.214328 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.214232    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-credential-socket\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.214489 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.214437    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-workload-certs\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.214549 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.214484    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-workload-socket\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.214549 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.214538    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istiod-ca-cert\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.216233 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.216204    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-envoy\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.216531 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.216513    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-podinfo\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.224139 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.224104    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4shxw\" (UniqueName: \"kubernetes.io/projected/cc67b0d3-1f45-4971-81ad-214aa8c1b844-kube-api-access-4shxw\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.224406 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.224383    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-token\") pod \"e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") " pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.313926 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.313831    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:12.375563 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.375529    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:12.469633 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.469607    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"]
Jul 16 16:09:12.471590 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:09:12.471564    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc67b0d3_1f45_4971_81ad_214aa8c1b844.slice/crio-c698f45f1608d92da7a7991906b481afdfd9efb25707c33ae9759d7071205487 WatchSource:0}: Error finding container c698f45f1608d92da7a7991906b481afdfd9efb25707c33ae9759d7071205487: Status 404 returned error can't find the container with id c698f45f1608d92da7a7991906b481afdfd9efb25707c33ae9759d7071205487
Jul 16 16:09:12.473874 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.473838    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:09:12.473975 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.473925    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:09:12.474014 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.473993    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:09:12.572234 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.572124    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd" event={"ID":"cc67b0d3-1f45-4971-81ad-214aa8c1b844","Type":"ContainerStarted","Data":"d2be1a90a2525e62813d13529381655338cd888abfe66b8758c0739efda53cbb"}
Jul 16 16:09:12.572234 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.572189    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd" event={"ID":"cc67b0d3-1f45-4971-81ad-214aa8c1b844","Type":"ContainerStarted","Data":"c698f45f1608d92da7a7991906b481afdfd9efb25707c33ae9759d7071205487"}
Jul 16 16:09:12.605696 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:12.605639    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd" podStartSLOduration=1.605618488 podStartE2EDuration="1.605618488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:09:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:09:12.604442139 +0000 UTC m=+2018.793306120" watchObservedRunningTime="2026-07-16 16:09:12.605618488 +0000 UTC m=+2018.794482516"
Jul 16 16:09:13.314384 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:13.314346    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:13.319216 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:13.319190    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:13.577619 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:13.577533    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:13.578464 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:13.578444    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:15.660204 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:15.660172    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:15.755648 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:15.755615    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:16.250845 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.250782    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:09:16.255735 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.255708    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-workload-socket\") pod \"7e838476-d512-4410-91e4-3ad2b7fdf56b\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") "
Jul 16 16:09:16.255868 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.255754    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e838476-d512-4410-91e4-3ad2b7fdf56b-istiod-ca-cert\") pod \"7e838476-d512-4410-91e4-3ad2b7fdf56b\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") "
Jul 16 16:09:16.255868 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.255786    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7xl5\" (UniqueName: \"kubernetes.io/projected/7e838476-d512-4410-91e4-3ad2b7fdf56b-kube-api-access-t7xl5\") pod \"7e838476-d512-4410-91e4-3ad2b7fdf56b\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") "
Jul 16 16:09:16.255868 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.255830    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-token\") pod \"7e838476-d512-4410-91e4-3ad2b7fdf56b\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") "
Jul 16 16:09:16.255868 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.255864    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-podinfo\") pod \"7e838476-d512-4410-91e4-3ad2b7fdf56b\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") "
Jul 16 16:09:16.256091 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.255893    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-credential-socket\") pod \"7e838476-d512-4410-91e4-3ad2b7fdf56b\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") "
Jul 16 16:09:16.256091 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.255914    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7e838476-d512-4410-91e4-3ad2b7fdf56b" (UID: "7e838476-d512-4410-91e4-3ad2b7fdf56b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:09:16.256091 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.255947    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-data\") pod \"7e838476-d512-4410-91e4-3ad2b7fdf56b\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") "
Jul 16 16:09:16.256091 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.255995    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-envoy\") pod \"7e838476-d512-4410-91e4-3ad2b7fdf56b\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") "
Jul 16 16:09:16.256091 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.256061    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-workload-certs\") pod \"7e838476-d512-4410-91e4-3ad2b7fdf56b\" (UID: \"7e838476-d512-4410-91e4-3ad2b7fdf56b\") "
Jul 16 16:09:16.256344 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.256248    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e838476-d512-4410-91e4-3ad2b7fdf56b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7e838476-d512-4410-91e4-3ad2b7fdf56b" (UID: "7e838476-d512-4410-91e4-3ad2b7fdf56b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 16:09:16.256344 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.256320    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7e838476-d512-4410-91e4-3ad2b7fdf56b" (UID: "7e838476-d512-4410-91e4-3ad2b7fdf56b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:09:16.256455 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.256376    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-workload-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:16.256455 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.256393    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e838476-d512-4410-91e4-3ad2b7fdf56b-istiod-ca-cert\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:16.256567 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.256477    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7e838476-d512-4410-91e4-3ad2b7fdf56b" (UID: "7e838476-d512-4410-91e4-3ad2b7fdf56b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:09:16.257000 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.256973    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-data" (OuterVolumeSpecName: "istio-data") pod "7e838476-d512-4410-91e4-3ad2b7fdf56b" (UID: "7e838476-d512-4410-91e4-3ad2b7fdf56b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:09:16.258393 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.258368    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e838476-d512-4410-91e4-3ad2b7fdf56b-kube-api-access-t7xl5" (OuterVolumeSpecName: "kube-api-access-t7xl5") pod "7e838476-d512-4410-91e4-3ad2b7fdf56b" (UID: "7e838476-d512-4410-91e4-3ad2b7fdf56b"). InnerVolumeSpecName "kube-api-access-t7xl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:09:16.258393 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.258373    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e838476-d512-4410-91e4-3ad2b7fdf56b" (UID: "7e838476-d512-4410-91e4-3ad2b7fdf56b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:09:16.258555 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.258390    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7e838476-d512-4410-91e4-3ad2b7fdf56b" (UID: "7e838476-d512-4410-91e4-3ad2b7fdf56b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:09:16.258555 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.258465    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7e838476-d512-4410-91e4-3ad2b7fdf56b" (UID: "7e838476-d512-4410-91e4-3ad2b7fdf56b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 16:09:16.356858 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.356820    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-workload-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:16.356858 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.356859    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7xl5\" (UniqueName: \"kubernetes.io/projected/7e838476-d512-4410-91e4-3ad2b7fdf56b-kube-api-access-t7xl5\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:16.357093 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.356875    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-token\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:16.357093 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.356888    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-podinfo\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:16.357093 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.356901    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-credential-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:16.357093 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.356914    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-data\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:16.357093 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.356926    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e838476-d512-4410-91e4-3ad2b7fdf56b-istio-envoy\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:09:16.595064 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.595030    2570 generic.go:356] "Generic (PLEG): container finished" podID="7e838476-d512-4410-91e4-3ad2b7fdf56b" containerID="8ee3622f634fbf1a7a4cd49672ccdf93d666f7c2d80496e97a2b55b0d3b939fd" exitCode=0
Jul 16 16:09:16.595231 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.595083    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"
Jul 16 16:09:16.595231 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.595112    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n" event={"ID":"7e838476-d512-4410-91e4-3ad2b7fdf56b","Type":"ContainerDied","Data":"8ee3622f634fbf1a7a4cd49672ccdf93d666f7c2d80496e97a2b55b0d3b939fd"}
Jul 16 16:09:16.595231 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.595165    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n" event={"ID":"7e838476-d512-4410-91e4-3ad2b7fdf56b","Type":"ContainerDied","Data":"e78976dfbc4c17756a044fb54e5df75faf02ab64a0cac515cae7bdf5fa592219"}
Jul 16 16:09:16.595231 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.595188    2570 scope.go:117] "RemoveContainer" containerID="8ee3622f634fbf1a7a4cd49672ccdf93d666f7c2d80496e97a2b55b0d3b939fd"
Jul 16 16:09:16.604359 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.604343    2570 scope.go:117] "RemoveContainer" containerID="724035b92a1f982ea7f96e0f4cd5878c8b3e71772ceba37a5258a61f7c740585"
Jul 16 16:09:16.612534 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.612516    2570 scope.go:117] "RemoveContainer" containerID="8ee3622f634fbf1a7a4cd49672ccdf93d666f7c2d80496e97a2b55b0d3b939fd"
Jul 16 16:09:16.612804 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:09:16.612786    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ee3622f634fbf1a7a4cd49672ccdf93d666f7c2d80496e97a2b55b0d3b939fd\": container with ID starting with 8ee3622f634fbf1a7a4cd49672ccdf93d666f7c2d80496e97a2b55b0d3b939fd not found: ID does not exist" containerID="8ee3622f634fbf1a7a4cd49672ccdf93d666f7c2d80496e97a2b55b0d3b939fd"
Jul 16 16:09:16.612854 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.612812    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ee3622f634fbf1a7a4cd49672ccdf93d666f7c2d80496e97a2b55b0d3b939fd"} err="failed to get container status \"8ee3622f634fbf1a7a4cd49672ccdf93d666f7c2d80496e97a2b55b0d3b939fd\": rpc error: code = NotFound desc = could not find container \"8ee3622f634fbf1a7a4cd49672ccdf93d666f7c2d80496e97a2b55b0d3b939fd\": container with ID starting with 8ee3622f634fbf1a7a4cd49672ccdf93d666f7c2d80496e97a2b55b0d3b939fd not found: ID does not exist"
Jul 16 16:09:16.612854 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.612835    2570 scope.go:117] "RemoveContainer" containerID="724035b92a1f982ea7f96e0f4cd5878c8b3e71772ceba37a5258a61f7c740585"
Jul 16 16:09:16.613122 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:09:16.613105    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"724035b92a1f982ea7f96e0f4cd5878c8b3e71772ceba37a5258a61f7c740585\": container with ID starting with 724035b92a1f982ea7f96e0f4cd5878c8b3e71772ceba37a5258a61f7c740585 not found: ID does not exist" containerID="724035b92a1f982ea7f96e0f4cd5878c8b3e71772ceba37a5258a61f7c740585"
Jul 16 16:09:16.613182 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.613132    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"724035b92a1f982ea7f96e0f4cd5878c8b3e71772ceba37a5258a61f7c740585"} err="failed to get container status \"724035b92a1f982ea7f96e0f4cd5878c8b3e71772ceba37a5258a61f7c740585\": rpc error: code = NotFound desc = could not find container \"724035b92a1f982ea7f96e0f4cd5878c8b3e71772ceba37a5258a61f7c740585\": container with ID starting with 724035b92a1f982ea7f96e0f4cd5878c8b3e71772ceba37a5258a61f7c740585 not found: ID does not exist"
Jul 16 16:09:16.621140 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.621118    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"]
Jul 16 16:09:16.634845 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:16.634824    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-22d3cd-openshift-default-6cdc69767c-h5k7n"]
Jul 16 16:09:18.267577 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:18.267542    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e838476-d512-4410-91e4-3ad2b7fdf56b" path="/var/lib/kubelet/pods/7e838476-d512-4410-91e4-3ad2b7fdf56b/volumes"
Jul 16 16:09:19.111995 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.111958    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:19.486865 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.486769    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:19.913376 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.913343    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"]
Jul 16 16:09:19.913959 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.913927    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e838476-d512-4410-91e4-3ad2b7fdf56b" containerName="istio-proxy"
Jul 16 16:09:19.914022 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.913960    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e838476-d512-4410-91e4-3ad2b7fdf56b" containerName="istio-proxy"
Jul 16 16:09:19.914065 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.914059    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e838476-d512-4410-91e4-3ad2b7fdf56b" containerName="istio-proxy"
Jul 16 16:09:19.914102 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.914072    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e838476-d512-4410-91e4-3ad2b7fdf56b" containerName="istio-proxy"
Jul 16 16:09:19.914102 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.914085    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a12ba9a8-4295-45b0-bf25-494a855300c9" containerName="istio-proxy"
Jul 16 16:09:19.918777 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.918757    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:19.921948 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.921916    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-590f9a-openshift-default-dockercfg-c2ms6\""
Jul 16 16:09:19.935700 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.935669    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"]
Jul 16 16:09:19.944842 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.944820    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:19.980462 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.980432    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-istio-envoy\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:19.980613 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.980470    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-workload-certs\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:19.980613 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.980491    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b99de390-ef87-4649-b524-32eb2f343632-istio-podinfo\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:19.980613 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.980536    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-istio-data\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:19.980613 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.980574    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-workload-socket\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:19.980613 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.980601    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b99de390-ef87-4649-b524-32eb2f343632-istiod-ca-cert\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:19.980780 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.980675    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmdk9\" (UniqueName: \"kubernetes.io/projected/b99de390-ef87-4649-b524-32eb2f343632-kube-api-access-bmdk9\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:19.980780 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.980741    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-credential-socket\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:19.980780 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:19.980767    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b99de390-ef87-4649-b524-32eb2f343632-istio-token\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.081517 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.081485    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-workload-certs\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.081730 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.081526    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b99de390-ef87-4649-b524-32eb2f343632-istio-podinfo\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.081730 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.081546    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-istio-data\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.081730 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.081566    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-workload-socket\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.081730 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.081586    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b99de390-ef87-4649-b524-32eb2f343632-istiod-ca-cert\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.081730 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.081625    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmdk9\" (UniqueName: \"kubernetes.io/projected/b99de390-ef87-4649-b524-32eb2f343632-kube-api-access-bmdk9\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.081730 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.081659    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-credential-socket\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.081730 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.081682    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b99de390-ef87-4649-b524-32eb2f343632-istio-token\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.082116 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.081737    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-istio-envoy\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.082116 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.082004    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-workload-certs\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.082116 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.082035    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-workload-socket\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.082116 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.082060    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-istio-data\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.082116 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.082106    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-credential-socket\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.082415 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.082389    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b99de390-ef87-4649-b524-32eb2f343632-istiod-ca-cert\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.084009 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.083981    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-istio-envoy\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.084203 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.084188    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b99de390-ef87-4649-b524-32eb2f343632-istio-podinfo\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.090795 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.090771    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b99de390-ef87-4649-b524-32eb2f343632-istio-token\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.092322 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.092295    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmdk9\" (UniqueName: \"kubernetes.io/projected/b99de390-ef87-4649-b524-32eb2f343632-kube-api-access-bmdk9\") pod \"e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") " pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.101178 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.101150    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:20.229097 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.228992    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:20.396777 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.396684    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"]
Jul 16 16:09:20.399654 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:09:20.399620    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb99de390_ef87_4649_b524_32eb2f343632.slice/crio-3dc49d96b8d5b1828dbbd35636cf8d1b83401debc9e53508aae614ab0883051c WatchSource:0}: Error finding container 3dc49d96b8d5b1828dbbd35636cf8d1b83401debc9e53508aae614ab0883051c: Status 404 returned error can't find the container with id 3dc49d96b8d5b1828dbbd35636cf8d1b83401debc9e53508aae614ab0883051c
Jul 16 16:09:20.402053 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.402019    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:09:20.402154 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.402106    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:09:20.402223 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.402149    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:09:20.620918 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.620872    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk" event={"ID":"b99de390-ef87-4649-b524-32eb2f343632","Type":"ContainerStarted","Data":"c5d18ed9a3daf4ee497871d8a3ab63d0cba4cd9c67b4d515da591dbc0bad01e6"}
Jul 16 16:09:20.621450 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.620963    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk" event={"ID":"b99de390-ef87-4649-b524-32eb2f343632","Type":"ContainerStarted","Data":"3dc49d96b8d5b1828dbbd35636cf8d1b83401debc9e53508aae614ab0883051c"}
Jul 16 16:09:20.648561 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:20.648484    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk" podStartSLOduration=1.648464148 podStartE2EDuration="1.648464148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:09:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:09:20.645382607 +0000 UTC m=+2026.834246586" watchObservedRunningTime="2026-07-16 16:09:20.648464148 +0000 UTC m=+2026.837328104"
Jul 16 16:09:21.229754 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:21.229713    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:21.231361 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:21.231327    2570 patch_prober.go:28] interesting pod/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 16 16:09:21.231592 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:21.231567    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk" podUID="b99de390-ef87-4649-b524-32eb2f343632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 16 16:09:22.236530 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:22.236490    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:22.633007 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:22.632974    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:22.635321 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:22.635085    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:22.712759 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:22.712721    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:22.784033 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:22.783998    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:22.936629 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:22.936436    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:23.037201 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:23.037171    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:23.578921 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:23.578891    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:23.679656 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:23.679628    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:27.222755 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:27.222702    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:28.579384 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:28.579349    2570 patch_prober.go:28] interesting pod/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 16 16:09:28.579798 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:28.579405    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd" podUID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 16 16:09:28.668512 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:28.668477    2570 generic.go:356] "Generic (PLEG): container finished" podID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerID="d2be1a90a2525e62813d13529381655338cd888abfe66b8758c0739efda53cbb" exitCode=0
Jul 16 16:09:28.668692 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:28.668551    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd" event={"ID":"cc67b0d3-1f45-4971-81ad-214aa8c1b844","Type":"ContainerDied","Data":"d2be1a90a2525e62813d13529381655338cd888abfe66b8758c0739efda53cbb"}
Jul 16 16:09:28.669029 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:28.669014    2570 scope.go:117] "RemoveContainer" containerID="d2be1a90a2525e62813d13529381655338cd888abfe66b8758c0739efda53cbb"
Jul 16 16:09:28.670052 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:28.670024    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:09:28.670145 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:28.670093    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:09:28.670145 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:28.670124    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:09:29.676034 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:29.675996    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd" event={"ID":"cc67b0d3-1f45-4971-81ad-214aa8c1b844","Type":"ContainerStarted","Data":"c6035b98b9182b25572d72feb2742c27b7e14b44c75c21b3ed576897ac3ecf86"}
Jul 16 16:09:30.314718 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:30.314678    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:30.315088 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:30.315067    2570 patch_prober.go:28] interesting pod/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 16 16:09:30.315152 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:30.315116    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd" podUID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 16 16:09:31.314561 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:31.314523    2570 patch_prober.go:28] interesting pod/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 16 16:09:31.315002 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:31.314586    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd" podUID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 16 16:09:32.314448 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:32.314411    2570 patch_prober.go:28] interesting pod/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 16 16:09:32.314636 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:32.314477    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd" podUID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 16 16:09:33.314400 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:33.314362    2570 patch_prober.go:28] interesting pod/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 16 16:09:33.314586 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:33.314416    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd" podUID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 16 16:09:34.319220 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:34.319186    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:34.319657 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:34.319482    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:34.320392 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:34.320376    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:09:36.369413 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:36.369379    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:36.465632 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:36.465542    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:37.634272 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:37.634232    2570 patch_prober.go:28] interesting pod/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 16 16:09:37.634721 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:37.634300    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk" podUID="b99de390-ef87-4649-b524-32eb2f343632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 16 16:09:37.722067 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:37.722031    2570 generic.go:356] "Generic (PLEG): container finished" podID="b99de390-ef87-4649-b524-32eb2f343632" containerID="c5d18ed9a3daf4ee497871d8a3ab63d0cba4cd9c67b4d515da591dbc0bad01e6" exitCode=0
Jul 16 16:09:37.722247 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:37.722121    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk" event={"ID":"b99de390-ef87-4649-b524-32eb2f343632","Type":"ContainerDied","Data":"c5d18ed9a3daf4ee497871d8a3ab63d0cba4cd9c67b4d515da591dbc0bad01e6"}
Jul 16 16:09:37.722556 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:37.722539    2570 scope.go:117] "RemoveContainer" containerID="c5d18ed9a3daf4ee497871d8a3ab63d0cba4cd9c67b4d515da591dbc0bad01e6"
Jul 16 16:09:37.723571 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:37.723545    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:09:37.723628 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:37.723611    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:09:37.723667 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:37.723648    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 16:09:38.749082 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:38.748168    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk" event={"ID":"b99de390-ef87-4649-b524-32eb2f343632","Type":"ContainerStarted","Data":"cadfac78571eabcd75cbc05cd1b704d4423f2b38f81d34a48ba5ce1c5312870a"}
Jul 16 16:09:38.973232 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:38.973201    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:39.229504 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:39.229449    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:39.229971 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:39.229917    2570 patch_prober.go:28] interesting pod/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 16 16:09:39.230106 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:39.229999    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk" podUID="b99de390-ef87-4649-b524-32eb2f343632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 16 16:09:40.230426 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:40.230385    2570 patch_prober.go:28] interesting pod/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 16 16:09:40.231064 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:40.231030    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk" podUID="b99de390-ef87-4649-b524-32eb2f343632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 16 16:09:41.230434 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:41.230386    2570 patch_prober.go:28] interesting pod/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 16 16:09:41.230970 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:41.230465    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk" podUID="b99de390-ef87-4649-b524-32eb2f343632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 16 16:09:42.230310 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:42.230264    2570 patch_prober.go:28] interesting pod/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 16 16:09:42.230556 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:42.230357    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk" podUID="b99de390-ef87-4649-b524-32eb2f343632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 16 16:09:43.235277 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:43.235245    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:43.235660 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:43.235525    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:43.236249 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:43.236232    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:09:45.478788 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:45.478758    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:45.579511 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:45.579481    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:59.122322 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.122237    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"]
Jul 16 16:09:59.122706 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.122609    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e838476-d512-4410-91e4-3ad2b7fdf56b" containerName="istio-proxy"
Jul 16 16:09:59.122706 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.122620    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e838476-d512-4410-91e4-3ad2b7fdf56b" containerName="istio-proxy"
Jul 16 16:09:59.126170 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.126153    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.129032 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.129004    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-590f9a\"/\"kube-root-ca.crt\""
Jul 16 16:09:59.129184 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.129004    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-590f9a\"/\"openshift-service-ca.crt\""
Jul 16 16:09:59.129959 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.129924    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-590f9a\"/\"default-dockercfg-mbnkt\""
Jul 16 16:09:59.130062 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.129994    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-590f9a\"/\"auth-test-model-590f9a-kserve-self-signed-certs\""
Jul 16 16:09:59.141947 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.141910    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"]
Jul 16 16:09:59.157317 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.157293    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:09:59.256346 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.256307    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbl9v\" (UniqueName: \"kubernetes.io/projected/f8855b83-3856-46af-b582-6250c62efb4c-kube-api-access-pbl9v\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.256346 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.256351    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-model-cache\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.256559 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.256408    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8855b83-3856-46af-b582-6250c62efb4c-tls-certs\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.256559 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.256463    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-dshm\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.256559 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.256481    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-home\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.256559 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.256550    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-tmp-dir\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.357595 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.357563    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbl9v\" (UniqueName: \"kubernetes.io/projected/f8855b83-3856-46af-b582-6250c62efb4c-kube-api-access-pbl9v\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.357795 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.357603    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-model-cache\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.357795 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.357628    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8855b83-3856-46af-b582-6250c62efb4c-tls-certs\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.357795 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.357662    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-dshm\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.358005 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.357796    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-home\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.358005 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.357883    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-tmp-dir\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.358079 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.358041    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-model-cache\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.358140 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.358104    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-home\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.358193 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.358151    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-tmp-dir\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.360007 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.359977    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-dshm\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.360222 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.360201    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8855b83-3856-46af-b582-6250c62efb4c-tls-certs\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.366003 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.365984    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbl9v\" (UniqueName: \"kubernetes.io/projected/f8855b83-3856-46af-b582-6250c62efb4c-kube-api-access-pbl9v\") pod \"auth-test-model-590f9a-kserve-669c899b99-s6jbk\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") " pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.442785 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.442692    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:09:59.615533 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.615474    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"]
Jul 16 16:09:59.617805 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:09:59.617781    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8855b83_3856_46af_b582_6250c62efb4c.slice/crio-e71b26bd068813ca94a6547a52af9773160b14370ddee6a445b0b4894240281b WatchSource:0}: Error finding container e71b26bd068813ca94a6547a52af9773160b14370ddee6a445b0b4894240281b: Status 404 returned error can't find the container with id e71b26bd068813ca94a6547a52af9773160b14370ddee6a445b0b4894240281b
Jul 16 16:09:59.858837 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.858803    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk" event={"ID":"f8855b83-3856-46af-b582-6250c62efb4c","Type":"ContainerStarted","Data":"f27d8cbf1daa502b74de63e0e85ac2f35048b206ec1791f4d7277ebc5bc0220f"}
Jul 16 16:09:59.858837 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.858843    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk" event={"ID":"f8855b83-3856-46af-b582-6250c62efb4c","Type":"ContainerStarted","Data":"e71b26bd068813ca94a6547a52af9773160b14370ddee6a445b0b4894240281b"}
Jul 16 16:09:59.882054 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:09:59.881998    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk" podStartSLOduration=0.881983911 podStartE2EDuration="881.983911ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:09:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:09:59.880269635 +0000 UTC m=+2066.069133590" watchObservedRunningTime="2026-07-16 16:09:59.881983911 +0000 UTC m=+2066.070847865"
Jul 16 16:10:04.815427 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:10:04.815389    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:10:08.048789 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:10:08.048755    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:10:09.443003 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:10:09.442968    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:10:09.443446 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:10:09.443017    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:10:09.455523 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:10:09.455491    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:10:09.917611 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:10:09.917583    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:10:34.316074 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:10:34.316044    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 16:10:34.319172 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:10:34.319151    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 16:11:48.835947 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:11:48.835891    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:12:00.901185 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:00.901147    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:12:01.811551 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:01.811520    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"]
Jul 16 16:12:01.827131 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:01.827100    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:12:16.831564 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:16.831521    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk" podUID="f8855b83-3856-46af-b582-6250c62efb4c" containerName="main" containerID="cri-o://f27d8cbf1daa502b74de63e0e85ac2f35048b206ec1791f4d7277ebc5bc0220f" gracePeriod=45
Jul 16 16:12:17.076687 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.076664    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:12:17.191335 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.191304    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8855b83-3856-46af-b582-6250c62efb4c-tls-certs\") pod \"f8855b83-3856-46af-b582-6250c62efb4c\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") "
Jul 16 16:12:17.191490 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.191366    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-model-cache\") pod \"f8855b83-3856-46af-b582-6250c62efb4c\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") "
Jul 16 16:12:17.191490 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.191425    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbl9v\" (UniqueName: \"kubernetes.io/projected/f8855b83-3856-46af-b582-6250c62efb4c-kube-api-access-pbl9v\") pod \"f8855b83-3856-46af-b582-6250c62efb4c\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") "
Jul 16 16:12:17.191490 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.191456    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-tmp-dir\") pod \"f8855b83-3856-46af-b582-6250c62efb4c\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") "
Jul 16 16:12:17.191639 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.191504    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-home\") pod \"f8855b83-3856-46af-b582-6250c62efb4c\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") "
Jul 16 16:12:17.191639 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.191541    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-dshm\") pod \"f8855b83-3856-46af-b582-6250c62efb4c\" (UID: \"f8855b83-3856-46af-b582-6250c62efb4c\") "
Jul 16 16:12:17.191752 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.191657    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-model-cache" (OuterVolumeSpecName: "model-cache") pod "f8855b83-3856-46af-b582-6250c62efb4c" (UID: "f8855b83-3856-46af-b582-6250c62efb4c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:12:17.191814 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.191774    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f8855b83-3856-46af-b582-6250c62efb4c" (UID: "f8855b83-3856-46af-b582-6250c62efb4c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:12:17.191814 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.191790    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-home" (OuterVolumeSpecName: "home") pod "f8855b83-3856-46af-b582-6250c62efb4c" (UID: "f8855b83-3856-46af-b582-6250c62efb4c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:12:17.191973 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.191926    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-model-cache\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:12:17.191973 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.191974    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-tmp-dir\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:12:17.192145 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.191987    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-home\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:12:17.193638 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.193615    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8855b83-3856-46af-b582-6250c62efb4c-kube-api-access-pbl9v" (OuterVolumeSpecName: "kube-api-access-pbl9v") pod "f8855b83-3856-46af-b582-6250c62efb4c" (UID: "f8855b83-3856-46af-b582-6250c62efb4c"). InnerVolumeSpecName "kube-api-access-pbl9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:12:17.193638 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.193624    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8855b83-3856-46af-b582-6250c62efb4c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f8855b83-3856-46af-b582-6250c62efb4c" (UID: "f8855b83-3856-46af-b582-6250c62efb4c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 16:12:17.193757 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.193620    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-dshm" (OuterVolumeSpecName: "dshm") pod "f8855b83-3856-46af-b582-6250c62efb4c" (UID: "f8855b83-3856-46af-b582-6250c62efb4c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:12:17.293399 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.293366    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbl9v\" (UniqueName: \"kubernetes.io/projected/f8855b83-3856-46af-b582-6250c62efb4c-kube-api-access-pbl9v\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:12:17.293399 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.293394    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8855b83-3856-46af-b582-6250c62efb4c-dshm\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:12:17.293399 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.293404    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8855b83-3856-46af-b582-6250c62efb4c-tls-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:12:17.505685 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.505594    2570 generic.go:356] "Generic (PLEG): container finished" podID="f8855b83-3856-46af-b582-6250c62efb4c" containerID="f27d8cbf1daa502b74de63e0e85ac2f35048b206ec1791f4d7277ebc5bc0220f" exitCode=0
Jul 16 16:12:17.505685 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.505637    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk" event={"ID":"f8855b83-3856-46af-b582-6250c62efb4c","Type":"ContainerDied","Data":"f27d8cbf1daa502b74de63e0e85ac2f35048b206ec1791f4d7277ebc5bc0220f"}
Jul 16 16:12:17.505685 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.505638    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"
Jul 16 16:12:17.505685 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.505677    2570 scope.go:117] "RemoveContainer" containerID="f27d8cbf1daa502b74de63e0e85ac2f35048b206ec1791f4d7277ebc5bc0220f"
Jul 16 16:12:17.506013 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.505667    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk" event={"ID":"f8855b83-3856-46af-b582-6250c62efb4c","Type":"ContainerDied","Data":"e71b26bd068813ca94a6547a52af9773160b14370ddee6a445b0b4894240281b"}
Jul 16 16:12:17.515006 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.514980    2570 scope.go:117] "RemoveContainer" containerID="f27d8cbf1daa502b74de63e0e85ac2f35048b206ec1791f4d7277ebc5bc0220f"
Jul 16 16:12:17.515263 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:12:17.515242    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f27d8cbf1daa502b74de63e0e85ac2f35048b206ec1791f4d7277ebc5bc0220f\": container with ID starting with f27d8cbf1daa502b74de63e0e85ac2f35048b206ec1791f4d7277ebc5bc0220f not found: ID does not exist" containerID="f27d8cbf1daa502b74de63e0e85ac2f35048b206ec1791f4d7277ebc5bc0220f"
Jul 16 16:12:17.515311 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.515275    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f27d8cbf1daa502b74de63e0e85ac2f35048b206ec1791f4d7277ebc5bc0220f"} err="failed to get container status \"f27d8cbf1daa502b74de63e0e85ac2f35048b206ec1791f4d7277ebc5bc0220f\": rpc error: code = NotFound desc = could not find container \"f27d8cbf1daa502b74de63e0e85ac2f35048b206ec1791f4d7277ebc5bc0220f\": container with ID starting with f27d8cbf1daa502b74de63e0e85ac2f35048b206ec1791f4d7277ebc5bc0220f not found: ID does not exist"
Jul 16 16:12:17.536714 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.536679    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"]
Jul 16 16:12:17.540633 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:17.540610    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/auth-test-model-590f9a-kserve-669c899b99-s6jbk"]
Jul 16 16:12:18.267158 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.267125    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8855b83-3856-46af-b582-6250c62efb4c" path="/var/lib/kubelet/pods/f8855b83-3856-46af-b582-6250c62efb4c/volumes"
Jul 16 16:12:18.379359 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.379323    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"]
Jul 16 16:12:18.379703 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.379691    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8855b83-3856-46af-b582-6250c62efb4c" containerName="main"
Jul 16 16:12:18.379746 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.379704    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8855b83-3856-46af-b582-6250c62efb4c" containerName="main"
Jul 16 16:12:18.379785 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.379762    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8855b83-3856-46af-b582-6250c62efb4c" containerName="main"
Jul 16 16:12:18.383860 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.383844    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.388502 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.388481    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-83e7d4\"/\"openshift-service-ca.crt\""
Jul 16 16:12:18.389623 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.389603    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-83e7d4\"/\"kube-root-ca.crt\""
Jul 16 16:12:18.389723 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.389609    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-83e7d4\"/\"default-dockercfg-hxqbc\""
Jul 16 16:12:18.389785 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.389729    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-83e7d4\"/\"sub-test-model-83e7d4-kserve-self-signed-certs\""
Jul 16 16:12:18.400510 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.400492    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"]
Jul 16 16:12:18.412145 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.412118    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:12:18.504972 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.504925    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-home\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.505111 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.505010    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-model-cache\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.505111 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.505032    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19e80e2f-010b-432e-bb2b-de688b46b3cf-tls-certs\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.505111 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.505065    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-dshm\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.505284 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.505103    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-tmp-dir\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.505284 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.505149    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvxc8\" (UniqueName: \"kubernetes.io/projected/19e80e2f-010b-432e-bb2b-de688b46b3cf-kube-api-access-jvxc8\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.605605 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.605578    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-home\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.605775 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.605635    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-model-cache\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.605775 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.605660    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19e80e2f-010b-432e-bb2b-de688b46b3cf-tls-certs\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.605775 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.605691    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-dshm\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.605775 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.605707    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-tmp-dir\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.605775 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.605734    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvxc8\" (UniqueName: \"kubernetes.io/projected/19e80e2f-010b-432e-bb2b-de688b46b3cf-kube-api-access-jvxc8\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.606070 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.606048    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-home\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.606130 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.606101    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-model-cache\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.606186 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.606140    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-tmp-dir\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.608147 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.608126    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-dshm\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.608446 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.608428    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19e80e2f-010b-432e-bb2b-de688b46b3cf-tls-certs\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.617308 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.617281    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvxc8\" (UniqueName: \"kubernetes.io/projected/19e80e2f-010b-432e-bb2b-de688b46b3cf-kube-api-access-jvxc8\") pod \"sub-test-model-83e7d4-kserve-5c67d7f477-57pkh\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") " pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.693173 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.693147    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:18.840673 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:18.840648    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"]
Jul 16 16:12:18.843211 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:12:18.843185    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19e80e2f_010b_432e_bb2b_de688b46b3cf.slice/crio-76022bf55a01e64bf0e048b81c2e5bcb9b00ba224b52b3659aec0077f1f95ceb WatchSource:0}: Error finding container 76022bf55a01e64bf0e048b81c2e5bcb9b00ba224b52b3659aec0077f1f95ceb: Status 404 returned error can't find the container with id 76022bf55a01e64bf0e048b81c2e5bcb9b00ba224b52b3659aec0077f1f95ceb
Jul 16 16:12:19.517248 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:19.517214    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh" event={"ID":"19e80e2f-010b-432e-bb2b-de688b46b3cf","Type":"ContainerStarted","Data":"1d8d5dd3da2fa6d8799c1de8a4058a1fe3cf93e6f9bda761e1cf3f4fffaa0eee"}
Jul 16 16:12:19.517721 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:19.517262    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh" event={"ID":"19e80e2f-010b-432e-bb2b-de688b46b3cf","Type":"ContainerStarted","Data":"76022bf55a01e64bf0e048b81c2e5bcb9b00ba224b52b3659aec0077f1f95ceb"}
Jul 16 16:12:19.544454 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:19.544411    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh" podStartSLOduration=1.544397155 podStartE2EDuration="1.544397155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:12:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:12:19.541589261 +0000 UTC m=+2205.730453215" watchObservedRunningTime="2026-07-16 16:12:19.544397155 +0000 UTC m=+2205.733261109"
Jul 16 16:12:19.720027 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:19.719996    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:12:28.693588 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:28.693548    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:28.694018 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:28.693628    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:28.706768 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:28.706739    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:29.578386 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:29.578356    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:12:54.891701 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:54.891660    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:12:54.979495 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:12:54.979460    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:13:42.146226 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:42.146196    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"]
Jul 16 16:13:42.164435 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:42.164404    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:13:43.067557 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:43.067511    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:13:57.165829 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.165763    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh" podUID="19e80e2f-010b-432e-bb2b-de688b46b3cf" containerName="main" containerID="cri-o://1d8d5dd3da2fa6d8799c1de8a4058a1fe3cf93e6f9bda761e1cf3f4fffaa0eee" gracePeriod=45
Jul 16 16:13:57.413540 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.413513    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:13:57.569485 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.569394    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvxc8\" (UniqueName: \"kubernetes.io/projected/19e80e2f-010b-432e-bb2b-de688b46b3cf-kube-api-access-jvxc8\") pod \"19e80e2f-010b-432e-bb2b-de688b46b3cf\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") "
Jul 16 16:13:57.569485 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.569457    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-tmp-dir\") pod \"19e80e2f-010b-432e-bb2b-de688b46b3cf\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") "
Jul 16 16:13:57.569724 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.569503    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-model-cache\") pod \"19e80e2f-010b-432e-bb2b-de688b46b3cf\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") "
Jul 16 16:13:57.569724 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.569520    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-home\") pod \"19e80e2f-010b-432e-bb2b-de688b46b3cf\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") "
Jul 16 16:13:57.569724 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.569587    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19e80e2f-010b-432e-bb2b-de688b46b3cf-tls-certs\") pod \"19e80e2f-010b-432e-bb2b-de688b46b3cf\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") "
Jul 16 16:13:57.569724 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.569607    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-dshm\") pod \"19e80e2f-010b-432e-bb2b-de688b46b3cf\" (UID: \"19e80e2f-010b-432e-bb2b-de688b46b3cf\") "
Jul 16 16:13:57.569904 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.569744    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "19e80e2f-010b-432e-bb2b-de688b46b3cf" (UID: "19e80e2f-010b-432e-bb2b-de688b46b3cf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:13:57.569904 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.569759    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-model-cache" (OuterVolumeSpecName: "model-cache") pod "19e80e2f-010b-432e-bb2b-de688b46b3cf" (UID: "19e80e2f-010b-432e-bb2b-de688b46b3cf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:13:57.569904 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.569835    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-home" (OuterVolumeSpecName: "home") pod "19e80e2f-010b-432e-bb2b-de688b46b3cf" (UID: "19e80e2f-010b-432e-bb2b-de688b46b3cf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:13:57.569904 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.569890    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-tmp-dir\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:13:57.569904 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.569901    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-model-cache\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:13:57.570113 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.569910    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-home\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:13:57.571598 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.571569    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19e80e2f-010b-432e-bb2b-de688b46b3cf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "19e80e2f-010b-432e-bb2b-de688b46b3cf" (UID: "19e80e2f-010b-432e-bb2b-de688b46b3cf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 16:13:57.571598 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.571582    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-dshm" (OuterVolumeSpecName: "dshm") pod "19e80e2f-010b-432e-bb2b-de688b46b3cf" (UID: "19e80e2f-010b-432e-bb2b-de688b46b3cf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:13:57.571739 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.571576    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19e80e2f-010b-432e-bb2b-de688b46b3cf-kube-api-access-jvxc8" (OuterVolumeSpecName: "kube-api-access-jvxc8") pod "19e80e2f-010b-432e-bb2b-de688b46b3cf" (UID: "19e80e2f-010b-432e-bb2b-de688b46b3cf"). InnerVolumeSpecName "kube-api-access-jvxc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:13:57.671119 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.671088    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19e80e2f-010b-432e-bb2b-de688b46b3cf-tls-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:13:57.671119 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.671115    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19e80e2f-010b-432e-bb2b-de688b46b3cf-dshm\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:13:57.671119 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.671124    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvxc8\" (UniqueName: \"kubernetes.io/projected/19e80e2f-010b-432e-bb2b-de688b46b3cf-kube-api-access-jvxc8\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:13:57.982078 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.982043    2570 generic.go:356] "Generic (PLEG): container finished" podID="19e80e2f-010b-432e-bb2b-de688b46b3cf" containerID="1d8d5dd3da2fa6d8799c1de8a4058a1fe3cf93e6f9bda761e1cf3f4fffaa0eee" exitCode=0
Jul 16 16:13:57.982250 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.982103    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"
Jul 16 16:13:57.982250 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.982136    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh" event={"ID":"19e80e2f-010b-432e-bb2b-de688b46b3cf","Type":"ContainerDied","Data":"1d8d5dd3da2fa6d8799c1de8a4058a1fe3cf93e6f9bda761e1cf3f4fffaa0eee"}
Jul 16 16:13:57.982250 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.982181    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh" event={"ID":"19e80e2f-010b-432e-bb2b-de688b46b3cf","Type":"ContainerDied","Data":"76022bf55a01e64bf0e048b81c2e5bcb9b00ba224b52b3659aec0077f1f95ceb"}
Jul 16 16:13:57.982250 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.982198    2570 scope.go:117] "RemoveContainer" containerID="1d8d5dd3da2fa6d8799c1de8a4058a1fe3cf93e6f9bda761e1cf3f4fffaa0eee"
Jul 16 16:13:57.990683 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.990664    2570 scope.go:117] "RemoveContainer" containerID="1d8d5dd3da2fa6d8799c1de8a4058a1fe3cf93e6f9bda761e1cf3f4fffaa0eee"
Jul 16 16:13:57.990913 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:13:57.990896    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d8d5dd3da2fa6d8799c1de8a4058a1fe3cf93e6f9bda761e1cf3f4fffaa0eee\": container with ID starting with 1d8d5dd3da2fa6d8799c1de8a4058a1fe3cf93e6f9bda761e1cf3f4fffaa0eee not found: ID does not exist" containerID="1d8d5dd3da2fa6d8799c1de8a4058a1fe3cf93e6f9bda761e1cf3f4fffaa0eee"
Jul 16 16:13:57.991003 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:57.990919    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d8d5dd3da2fa6d8799c1de8a4058a1fe3cf93e6f9bda761e1cf3f4fffaa0eee"} err="failed to get container status \"1d8d5dd3da2fa6d8799c1de8a4058a1fe3cf93e6f9bda761e1cf3f4fffaa0eee\": rpc error: code = NotFound desc = could not find container \"1d8d5dd3da2fa6d8799c1de8a4058a1fe3cf93e6f9bda761e1cf3f4fffaa0eee\": container with ID starting with 1d8d5dd3da2fa6d8799c1de8a4058a1fe3cf93e6f9bda761e1cf3f4fffaa0eee not found: ID does not exist"
Jul 16 16:13:58.007881 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:58.007859    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"]
Jul 16 16:13:58.015536 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:58.015511    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/sub-test-model-83e7d4-kserve-5c67d7f477-57pkh"]
Jul 16 16:13:58.268382 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:58.268290    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19e80e2f-010b-432e-bb2b-de688b46b3cf" path="/var/lib/kubelet/pods/19e80e2f-010b-432e-bb2b-de688b46b3cf/volumes"
Jul 16 16:13:59.028082 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.028050    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"]
Jul 16 16:13:59.028418 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.028406    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19e80e2f-010b-432e-bb2b-de688b46b3cf" containerName="main"
Jul 16 16:13:59.028462 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.028419    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="19e80e2f-010b-432e-bb2b-de688b46b3cf" containerName="main"
Jul 16 16:13:59.028497 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.028477    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="19e80e2f-010b-432e-bb2b-de688b46b3cf" containerName="main"
Jul 16 16:13:59.032791 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.032774    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.035341 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.035321    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-83e7d4\"/\"kube-root-ca.crt\""
Jul 16 16:13:59.036535 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.036519    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-83e7d4\"/\"rate-test-model-83e7d4-kserve-self-signed-certs\""
Jul 16 16:13:59.036627 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.036569    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-83e7d4\"/\"default-dockercfg-hxqbc\""
Jul 16 16:13:59.036692 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.036621    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-83e7d4\"/\"openshift-service-ca.crt\""
Jul 16 16:13:59.040842 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.040822    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"]
Jul 16 16:13:59.105804 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.105770    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:13:59.184785 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.184754    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-dshm\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.184986 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.184795    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50934403-ff3d-4f59-a08d-221a3456e9c1-tls-certs\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.184986 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.184814    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv5lw\" (UniqueName: \"kubernetes.io/projected/50934403-ff3d-4f59-a08d-221a3456e9c1-kube-api-access-vv5lw\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.184986 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.184834    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-tmp-dir\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.184986 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.184883    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-home\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.184986 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.184955    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-model-cache\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.286685 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.286591    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-home\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.287234 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.287184    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-home\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.287368 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.287300    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-model-cache\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.287432 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.287377    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-dshm\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.287492 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.287428    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50934403-ff3d-4f59-a08d-221a3456e9c1-tls-certs\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.287492 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.287453    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv5lw\" (UniqueName: \"kubernetes.io/projected/50934403-ff3d-4f59-a08d-221a3456e9c1-kube-api-access-vv5lw\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.287492 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.287484    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-tmp-dir\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.289276 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.289253    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-model-cache\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.289477 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.289456    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-tmp-dir\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.292159 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.292136    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50934403-ff3d-4f59-a08d-221a3456e9c1-tls-certs\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.296427 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.296381    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-dshm\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.297643 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.297620    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv5lw\" (UniqueName: \"kubernetes.io/projected/50934403-ff3d-4f59-a08d-221a3456e9c1-kube-api-access-vv5lw\") pod \"rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") " pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.342498 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.342475    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:13:59.477348 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.477324    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"]
Jul 16 16:13:59.479626 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:13:59.479600    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50934403_ff3d_4f59_a08d_221a3456e9c1.slice/crio-2d6b9b7a81a9d438d161188ab0c816f3814d4f99412e59f2ed04b1d99274e0ae WatchSource:0}: Error finding container 2d6b9b7a81a9d438d161188ab0c816f3814d4f99412e59f2ed04b1d99274e0ae: Status 404 returned error can't find the container with id 2d6b9b7a81a9d438d161188ab0c816f3814d4f99412e59f2ed04b1d99274e0ae
Jul 16 16:13:59.994061 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.994021    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7" event={"ID":"50934403-ff3d-4f59-a08d-221a3456e9c1","Type":"ContainerStarted","Data":"61b1d8dbb6202295d900394f925f3abc4aa8e1e5f9cfc66b21e540d3d2fb5642"}
Jul 16 16:13:59.994241 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:13:59.994069    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7" event={"ID":"50934403-ff3d-4f59-a08d-221a3456e9c1","Type":"ContainerStarted","Data":"2d6b9b7a81a9d438d161188ab0c816f3814d4f99412e59f2ed04b1d99274e0ae"}
Jul 16 16:14:00.019575 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:14:00.019525    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7" podStartSLOduration=1.019510716 podStartE2EDuration="1.019510716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:14:00.01657969 +0000 UTC m=+2306.205443643" watchObservedRunningTime="2026-07-16 16:14:00.019510716 +0000 UTC m=+2306.208374669"
Jul 16 16:14:09.342827 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:14:09.342791    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:14:09.343370 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:14:09.343094    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:14:09.354951 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:14:09.354906    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:14:10.053717 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:14:10.053688    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:14:17.150916 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:14:17.150884    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:14:17.744058 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:14:17.744021    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:14:28.888204 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:14:28.888123    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:14:37.173946 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:14:37.173897    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:14:37.274057 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:14:37.274021    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:14:46.297517 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:14:46.297482    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:14:48.950711 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:14:48.950671    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:01.879588 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:01.879556    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"]
Jul 16 16:15:01.891102 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:01.891073    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:02.787968 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:02.787924    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:16.899647 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:16.899601    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7" podUID="50934403-ff3d-4f59-a08d-221a3456e9c1" containerName="main" containerID="cri-o://61b1d8dbb6202295d900394f925f3abc4aa8e1e5f9cfc66b21e540d3d2fb5642" gracePeriod=45
Jul 16 16:15:17.162139 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.162073    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:15:17.282039 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.281998    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-model-cache\") pod \"50934403-ff3d-4f59-a08d-221a3456e9c1\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") "
Jul 16 16:15:17.282039 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.282046    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-home\") pod \"50934403-ff3d-4f59-a08d-221a3456e9c1\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") "
Jul 16 16:15:17.282299 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.282079    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50934403-ff3d-4f59-a08d-221a3456e9c1-tls-certs\") pod \"50934403-ff3d-4f59-a08d-221a3456e9c1\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") "
Jul 16 16:15:17.282299 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.282108    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-dshm\") pod \"50934403-ff3d-4f59-a08d-221a3456e9c1\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") "
Jul 16 16:15:17.282299 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.282147    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-tmp-dir\") pod \"50934403-ff3d-4f59-a08d-221a3456e9c1\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") "
Jul 16 16:15:17.282299 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.282210    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv5lw\" (UniqueName: \"kubernetes.io/projected/50934403-ff3d-4f59-a08d-221a3456e9c1-kube-api-access-vv5lw\") pod \"50934403-ff3d-4f59-a08d-221a3456e9c1\" (UID: \"50934403-ff3d-4f59-a08d-221a3456e9c1\") "
Jul 16 16:15:17.282519 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.282307    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-model-cache" (OuterVolumeSpecName: "model-cache") pod "50934403-ff3d-4f59-a08d-221a3456e9c1" (UID: "50934403-ff3d-4f59-a08d-221a3456e9c1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:15:17.282519 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.282334    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-home" (OuterVolumeSpecName: "home") pod "50934403-ff3d-4f59-a08d-221a3456e9c1" (UID: "50934403-ff3d-4f59-a08d-221a3456e9c1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:15:17.282519 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.282440    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "50934403-ff3d-4f59-a08d-221a3456e9c1" (UID: "50934403-ff3d-4f59-a08d-221a3456e9c1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:15:17.282664 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.282635    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-model-cache\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:17.282664 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.282657    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-home\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:17.282737 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.282666    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-tmp-dir\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:17.284199 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.284178    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50934403-ff3d-4f59-a08d-221a3456e9c1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "50934403-ff3d-4f59-a08d-221a3456e9c1" (UID: "50934403-ff3d-4f59-a08d-221a3456e9c1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 16:15:17.284619 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.284599    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-dshm" (OuterVolumeSpecName: "dshm") pod "50934403-ff3d-4f59-a08d-221a3456e9c1" (UID: "50934403-ff3d-4f59-a08d-221a3456e9c1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:15:17.284697 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.284629    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50934403-ff3d-4f59-a08d-221a3456e9c1-kube-api-access-vv5lw" (OuterVolumeSpecName: "kube-api-access-vv5lw") pod "50934403-ff3d-4f59-a08d-221a3456e9c1" (UID: "50934403-ff3d-4f59-a08d-221a3456e9c1"). InnerVolumeSpecName "kube-api-access-vv5lw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:15:17.350751 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.350721    2570 generic.go:356] "Generic (PLEG): container finished" podID="50934403-ff3d-4f59-a08d-221a3456e9c1" containerID="61b1d8dbb6202295d900394f925f3abc4aa8e1e5f9cfc66b21e540d3d2fb5642" exitCode=0
Jul 16 16:15:17.350918 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.350769    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"
Jul 16 16:15:17.350918 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.350813    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7" event={"ID":"50934403-ff3d-4f59-a08d-221a3456e9c1","Type":"ContainerDied","Data":"61b1d8dbb6202295d900394f925f3abc4aa8e1e5f9cfc66b21e540d3d2fb5642"}
Jul 16 16:15:17.350918 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.350855    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7" event={"ID":"50934403-ff3d-4f59-a08d-221a3456e9c1","Type":"ContainerDied","Data":"2d6b9b7a81a9d438d161188ab0c816f3814d4f99412e59f2ed04b1d99274e0ae"}
Jul 16 16:15:17.350918 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.350871    2570 scope.go:117] "RemoveContainer" containerID="61b1d8dbb6202295d900394f925f3abc4aa8e1e5f9cfc66b21e540d3d2fb5642"
Jul 16 16:15:17.359754 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.359737    2570 scope.go:117] "RemoveContainer" containerID="61b1d8dbb6202295d900394f925f3abc4aa8e1e5f9cfc66b21e540d3d2fb5642"
Jul 16 16:15:17.360075 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:15:17.360054    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61b1d8dbb6202295d900394f925f3abc4aa8e1e5f9cfc66b21e540d3d2fb5642\": container with ID starting with 61b1d8dbb6202295d900394f925f3abc4aa8e1e5f9cfc66b21e540d3d2fb5642 not found: ID does not exist" containerID="61b1d8dbb6202295d900394f925f3abc4aa8e1e5f9cfc66b21e540d3d2fb5642"
Jul 16 16:15:17.360164 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.360084    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61b1d8dbb6202295d900394f925f3abc4aa8e1e5f9cfc66b21e540d3d2fb5642"} err="failed to get container status \"61b1d8dbb6202295d900394f925f3abc4aa8e1e5f9cfc66b21e540d3d2fb5642\": rpc error: code = NotFound desc = could not find container \"61b1d8dbb6202295d900394f925f3abc4aa8e1e5f9cfc66b21e540d3d2fb5642\": container with ID starting with 61b1d8dbb6202295d900394f925f3abc4aa8e1e5f9cfc66b21e540d3d2fb5642 not found: ID does not exist"
Jul 16 16:15:17.383904 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.383880    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50934403-ff3d-4f59-a08d-221a3456e9c1-tls-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:17.383904 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.383900    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50934403-ff3d-4f59-a08d-221a3456e9c1-dshm\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:17.384067 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.383909    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv5lw\" (UniqueName: \"kubernetes.io/projected/50934403-ff3d-4f59-a08d-221a3456e9c1-kube-api-access-vv5lw\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:17.387898 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.387876    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"]
Jul 16 16:15:17.400887 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:17.400856    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/rate-test-model-83e7d4-kserve-8777bbfc5-sfdw7"]
Jul 16 16:15:18.267581 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:18.267551    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50934403-ff3d-4f59-a08d-221a3456e9c1" path="/var/lib/kubelet/pods/50934403-ff3d-4f59-a08d-221a3456e9c1/volumes"
Jul 16 16:15:22.607613 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:22.607576    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:23.089591 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:23.089557    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:23.215662 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:23.215628    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:23.319629 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:23.319594    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:23.528447 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:23.528370    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:29.852576 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:29.852547    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:29.956117 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:29.956071    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:30.155894 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:30.155806    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"]
Jul 16 16:15:30.156066 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:30.156043    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd" podUID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerName="istio-proxy" containerID="cri-o://c6035b98b9182b25572d72feb2742c27b7e14b44c75c21b3ed576897ac3ecf86" gracePeriod=30
Jul 16 16:15:30.179615 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:30.179582    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:30.438854 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:30.438761    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:30.605744 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:30.605712    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:31.036778 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:31.036736    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:31.135600 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:31.135568    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:31.238271 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:31.238237    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:31.352403 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:31.352362    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:34.322230 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:34.322193    2570 patch_prober.go:28] interesting pod/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 16:15:34.322633 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:34.322253    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd" podUID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 16:15:34.342581 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:34.342554    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 16:15:34.346925 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:34.346904    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sltmp_fb36e980-e40a-42e8-b1db-f032b02d683a/ovn-acl-logging/0.log"
Jul 16 16:15:35.415112 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.415080    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:15:35.452436 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.452339    2570 generic.go:356] "Generic (PLEG): container finished" podID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerID="c6035b98b9182b25572d72feb2742c27b7e14b44c75c21b3ed576897ac3ecf86" exitCode=0
Jul 16 16:15:35.452436 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.452386    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"
Jul 16 16:15:35.452436 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.452426    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd" event={"ID":"cc67b0d3-1f45-4971-81ad-214aa8c1b844","Type":"ContainerDied","Data":"c6035b98b9182b25572d72feb2742c27b7e14b44c75c21b3ed576897ac3ecf86"}
Jul 16 16:15:35.452691 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.452484    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd" event={"ID":"cc67b0d3-1f45-4971-81ad-214aa8c1b844","Type":"ContainerDied","Data":"c698f45f1608d92da7a7991906b481afdfd9efb25707c33ae9759d7071205487"}
Jul 16 16:15:35.452691 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.452515    2570 scope.go:117] "RemoveContainer" containerID="c6035b98b9182b25572d72feb2742c27b7e14b44c75c21b3ed576897ac3ecf86"
Jul 16 16:15:35.461552 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.461531    2570 scope.go:117] "RemoveContainer" containerID="d2be1a90a2525e62813d13529381655338cd888abfe66b8758c0739efda53cbb"
Jul 16 16:15:35.469704 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.469684    2570 scope.go:117] "RemoveContainer" containerID="c6035b98b9182b25572d72feb2742c27b7e14b44c75c21b3ed576897ac3ecf86"
Jul 16 16:15:35.470106 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:15:35.470086    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6035b98b9182b25572d72feb2742c27b7e14b44c75c21b3ed576897ac3ecf86\": container with ID starting with c6035b98b9182b25572d72feb2742c27b7e14b44c75c21b3ed576897ac3ecf86 not found: ID does not exist" containerID="c6035b98b9182b25572d72feb2742c27b7e14b44c75c21b3ed576897ac3ecf86"
Jul 16 16:15:35.470186 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.470116    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6035b98b9182b25572d72feb2742c27b7e14b44c75c21b3ed576897ac3ecf86"} err="failed to get container status \"c6035b98b9182b25572d72feb2742c27b7e14b44c75c21b3ed576897ac3ecf86\": rpc error: code = NotFound desc = could not find container \"c6035b98b9182b25572d72feb2742c27b7e14b44c75c21b3ed576897ac3ecf86\": container with ID starting with c6035b98b9182b25572d72feb2742c27b7e14b44c75c21b3ed576897ac3ecf86 not found: ID does not exist"
Jul 16 16:15:35.470186 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.470135    2570 scope.go:117] "RemoveContainer" containerID="d2be1a90a2525e62813d13529381655338cd888abfe66b8758c0739efda53cbb"
Jul 16 16:15:35.470423 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:15:35.470404    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2be1a90a2525e62813d13529381655338cd888abfe66b8758c0739efda53cbb\": container with ID starting with d2be1a90a2525e62813d13529381655338cd888abfe66b8758c0739efda53cbb not found: ID does not exist" containerID="d2be1a90a2525e62813d13529381655338cd888abfe66b8758c0739efda53cbb"
Jul 16 16:15:35.470471 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.470429    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2be1a90a2525e62813d13529381655338cd888abfe66b8758c0739efda53cbb"} err="failed to get container status \"d2be1a90a2525e62813d13529381655338cd888abfe66b8758c0739efda53cbb\": rpc error: code = NotFound desc = could not find container \"d2be1a90a2525e62813d13529381655338cd888abfe66b8758c0739efda53cbb\": container with ID starting with d2be1a90a2525e62813d13529381655338cd888abfe66b8758c0739efda53cbb not found: ID does not exist"
Jul 16 16:15:35.556274 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.556238    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-data\") pod \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") "
Jul 16 16:15:35.556482 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.556294    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istiod-ca-cert\") pod \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") "
Jul 16 16:15:35.556482 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.556332    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-workload-socket\") pod \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") "
Jul 16 16:15:35.556482 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.556367    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-token\") pod \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") "
Jul 16 16:15:35.556659 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.556500    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-workload-certs\") pod \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") "
Jul 16 16:15:35.556659 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.556549    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-credential-socket\") pod \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") "
Jul 16 16:15:35.556659 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.556588    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-envoy\") pod \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") "
Jul 16 16:15:35.556659 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.556599    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cc67b0d3-1f45-4971-81ad-214aa8c1b844" (UID: "cc67b0d3-1f45-4971-81ad-214aa8c1b844"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:15:35.556659 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.556627    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4shxw\" (UniqueName: \"kubernetes.io/projected/cc67b0d3-1f45-4971-81ad-214aa8c1b844-kube-api-access-4shxw\") pod \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") "
Jul 16 16:15:35.556918 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.556675    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-podinfo\") pod \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\" (UID: \"cc67b0d3-1f45-4971-81ad-214aa8c1b844\") "
Jul 16 16:15:35.556918 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.556766    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cc67b0d3-1f45-4971-81ad-214aa8c1b844" (UID: "cc67b0d3-1f45-4971-81ad-214aa8c1b844"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:15:35.556918 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.556796    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cc67b0d3-1f45-4971-81ad-214aa8c1b844" (UID: "cc67b0d3-1f45-4971-81ad-214aa8c1b844"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:15:35.556918 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.556833    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cc67b0d3-1f45-4971-81ad-214aa8c1b844" (UID: "cc67b0d3-1f45-4971-81ad-214aa8c1b844"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 16:15:35.557153 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.557035    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istiod-ca-cert\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:35.557153 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.557056    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-workload-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:35.557153 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.557068    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-workload-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:35.557153 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.557080    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-credential-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:35.557359 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.557238    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-data" (OuterVolumeSpecName: "istio-data") pod "cc67b0d3-1f45-4971-81ad-214aa8c1b844" (UID: "cc67b0d3-1f45-4971-81ad-214aa8c1b844"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:15:35.558685 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.558658    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-token" (OuterVolumeSpecName: "istio-token") pod "cc67b0d3-1f45-4971-81ad-214aa8c1b844" (UID: "cc67b0d3-1f45-4971-81ad-214aa8c1b844"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:15:35.558800 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.558781    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc67b0d3-1f45-4971-81ad-214aa8c1b844-kube-api-access-4shxw" (OuterVolumeSpecName: "kube-api-access-4shxw") pod "cc67b0d3-1f45-4971-81ad-214aa8c1b844" (UID: "cc67b0d3-1f45-4971-81ad-214aa8c1b844"). InnerVolumeSpecName "kube-api-access-4shxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:15:35.559250 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.559233    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cc67b0d3-1f45-4971-81ad-214aa8c1b844" (UID: "cc67b0d3-1f45-4971-81ad-214aa8c1b844"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 16:15:35.559304 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.559244    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cc67b0d3-1f45-4971-81ad-214aa8c1b844" (UID: "cc67b0d3-1f45-4971-81ad-214aa8c1b844"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:15:35.657831 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.657789    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-podinfo\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:35.657831 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.657828    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-data\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:35.657831 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.657843    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-token\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:35.658111 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.657857    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc67b0d3-1f45-4971-81ad-214aa8c1b844-istio-envoy\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:35.658111 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.657873    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4shxw\" (UniqueName: \"kubernetes.io/projected/cc67b0d3-1f45-4971-81ad-214aa8c1b844-kube-api-access-4shxw\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:35.776777 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.776745    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"]
Jul 16 16:15:35.781816 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:35.781787    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-83e7d4-openshift-default-5cc5ffbbd7-gwbhd"]
Jul 16 16:15:36.266866 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:36.266833    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" path="/var/lib/kubelet/pods/cc67b0d3-1f45-4971-81ad-214aa8c1b844/volumes"
Jul 16 16:15:37.414889 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:37.414848    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:37.519971 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:37.519916    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:37.719000 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:37.718887    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"]
Jul 16 16:15:37.719237 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:37.719195    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk" podUID="b99de390-ef87-4649-b524-32eb2f343632" containerName="istio-proxy" containerID="cri-o://cadfac78571eabcd75cbc05cd1b704d4423f2b38f81d34a48ba5ce1c5312870a" gracePeriod=30
Jul 16 16:15:37.732473 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:37.732443    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:37.990042 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:37.989939    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:42.985277 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:42.985199    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:15:43.127109 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.127068    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-istio-envoy\") pod \"b99de390-ef87-4649-b524-32eb2f343632\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") "
Jul 16 16:15:43.127109 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.127110    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-istio-data\") pod \"b99de390-ef87-4649-b524-32eb2f343632\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") "
Jul 16 16:15:43.127374 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.127128    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b99de390-ef87-4649-b524-32eb2f343632-istio-token\") pod \"b99de390-ef87-4649-b524-32eb2f343632\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") "
Jul 16 16:15:43.127374 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.127147    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-credential-socket\") pod \"b99de390-ef87-4649-b524-32eb2f343632\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") "
Jul 16 16:15:43.127374 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.127182    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b99de390-ef87-4649-b524-32eb2f343632-istio-podinfo\") pod \"b99de390-ef87-4649-b524-32eb2f343632\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") "
Jul 16 16:15:43.127374 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.127229    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmdk9\" (UniqueName: \"kubernetes.io/projected/b99de390-ef87-4649-b524-32eb2f343632-kube-api-access-bmdk9\") pod \"b99de390-ef87-4649-b524-32eb2f343632\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") "
Jul 16 16:15:43.127374 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.127270    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b99de390-ef87-4649-b524-32eb2f343632-istiod-ca-cert\") pod \"b99de390-ef87-4649-b524-32eb2f343632\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") "
Jul 16 16:15:43.127374 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.127338    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-workload-socket\") pod \"b99de390-ef87-4649-b524-32eb2f343632\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") "
Jul 16 16:15:43.127374 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.127362    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-workload-certs\") pod \"b99de390-ef87-4649-b524-32eb2f343632\" (UID: \"b99de390-ef87-4649-b524-32eb2f343632\") "
Jul 16 16:15:43.127738 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.127507    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b99de390-ef87-4649-b524-32eb2f343632" (UID: "b99de390-ef87-4649-b524-32eb2f343632"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:15:43.127738 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.127724    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-credential-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:43.127848 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.127720    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b99de390-ef87-4649-b524-32eb2f343632" (UID: "b99de390-ef87-4649-b524-32eb2f343632"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:15:43.127995 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.127975    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b99de390-ef87-4649-b524-32eb2f343632" (UID: "b99de390-ef87-4649-b524-32eb2f343632"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:15:43.128069 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.128007    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b99de390-ef87-4649-b524-32eb2f343632-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b99de390-ef87-4649-b524-32eb2f343632" (UID: "b99de390-ef87-4649-b524-32eb2f343632"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 16:15:43.128215 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.128185    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-istio-data" (OuterVolumeSpecName: "istio-data") pod "b99de390-ef87-4649-b524-32eb2f343632" (UID: "b99de390-ef87-4649-b524-32eb2f343632"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:15:43.129537 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.129507    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b99de390-ef87-4649-b524-32eb2f343632-istio-token" (OuterVolumeSpecName: "istio-token") pod "b99de390-ef87-4649-b524-32eb2f343632" (UID: "b99de390-ef87-4649-b524-32eb2f343632"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:15:43.129537 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.129532    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b99de390-ef87-4649-b524-32eb2f343632" (UID: "b99de390-ef87-4649-b524-32eb2f343632"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 16:15:43.129666 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.129543    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b99de390-ef87-4649-b524-32eb2f343632-kube-api-access-bmdk9" (OuterVolumeSpecName: "kube-api-access-bmdk9") pod "b99de390-ef87-4649-b524-32eb2f343632" (UID: "b99de390-ef87-4649-b524-32eb2f343632"). InnerVolumeSpecName "kube-api-access-bmdk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 16:15:43.129666 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.129616    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b99de390-ef87-4649-b524-32eb2f343632-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b99de390-ef87-4649-b524-32eb2f343632" (UID: "b99de390-ef87-4649-b524-32eb2f343632"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 16:15:43.228437 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.228397    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-istio-envoy\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:43.228437 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.228430    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-istio-data\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:43.228437 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.228438    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b99de390-ef87-4649-b524-32eb2f343632-istio-token\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:43.228437 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.228446    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b99de390-ef87-4649-b524-32eb2f343632-istio-podinfo\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:43.228708 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.228455    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmdk9\" (UniqueName: \"kubernetes.io/projected/b99de390-ef87-4649-b524-32eb2f343632-kube-api-access-bmdk9\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:43.228708 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.228464    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b99de390-ef87-4649-b524-32eb2f343632-istiod-ca-cert\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:43.228708 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.228471    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-workload-socket\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:43.228708 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.228478    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b99de390-ef87-4649-b524-32eb2f343632-workload-certs\") on node \"ip-10-0-141-84.ec2.internal\" DevicePath \"\""
Jul 16 16:15:43.501123 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.501084    2570 generic.go:356] "Generic (PLEG): container finished" podID="b99de390-ef87-4649-b524-32eb2f343632" containerID="cadfac78571eabcd75cbc05cd1b704d4423f2b38f81d34a48ba5ce1c5312870a" exitCode=0
Jul 16 16:15:43.501310 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.501140    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"
Jul 16 16:15:43.501310 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.501169    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk" event={"ID":"b99de390-ef87-4649-b524-32eb2f343632","Type":"ContainerDied","Data":"cadfac78571eabcd75cbc05cd1b704d4423f2b38f81d34a48ba5ce1c5312870a"}
Jul 16 16:15:43.501310 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.501226    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk" event={"ID":"b99de390-ef87-4649-b524-32eb2f343632","Type":"ContainerDied","Data":"3dc49d96b8d5b1828dbbd35636cf8d1b83401debc9e53508aae614ab0883051c"}
Jul 16 16:15:43.501310 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.501249    2570 scope.go:117] "RemoveContainer" containerID="cadfac78571eabcd75cbc05cd1b704d4423f2b38f81d34a48ba5ce1c5312870a"
Jul 16 16:15:43.510516 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.510500    2570 scope.go:117] "RemoveContainer" containerID="c5d18ed9a3daf4ee497871d8a3ab63d0cba4cd9c67b4d515da591dbc0bad01e6"
Jul 16 16:15:43.518389 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.518368    2570 scope.go:117] "RemoveContainer" containerID="cadfac78571eabcd75cbc05cd1b704d4423f2b38f81d34a48ba5ce1c5312870a"
Jul 16 16:15:43.518613 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:15:43.518595    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cadfac78571eabcd75cbc05cd1b704d4423f2b38f81d34a48ba5ce1c5312870a\": container with ID starting with cadfac78571eabcd75cbc05cd1b704d4423f2b38f81d34a48ba5ce1c5312870a not found: ID does not exist" containerID="cadfac78571eabcd75cbc05cd1b704d4423f2b38f81d34a48ba5ce1c5312870a"
Jul 16 16:15:43.518674 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.518627    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cadfac78571eabcd75cbc05cd1b704d4423f2b38f81d34a48ba5ce1c5312870a"} err="failed to get container status \"cadfac78571eabcd75cbc05cd1b704d4423f2b38f81d34a48ba5ce1c5312870a\": rpc error: code = NotFound desc = could not find container \"cadfac78571eabcd75cbc05cd1b704d4423f2b38f81d34a48ba5ce1c5312870a\": container with ID starting with cadfac78571eabcd75cbc05cd1b704d4423f2b38f81d34a48ba5ce1c5312870a not found: ID does not exist"
Jul 16 16:15:43.518674 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.518653    2570 scope.go:117] "RemoveContainer" containerID="c5d18ed9a3daf4ee497871d8a3ab63d0cba4cd9c67b4d515da591dbc0bad01e6"
Jul 16 16:15:43.518921 ip-10-0-141-84 kubenswrapper[2570]: E0716 16:15:43.518905    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5d18ed9a3daf4ee497871d8a3ab63d0cba4cd9c67b4d515da591dbc0bad01e6\": container with ID starting with c5d18ed9a3daf4ee497871d8a3ab63d0cba4cd9c67b4d515da591dbc0bad01e6 not found: ID does not exist" containerID="c5d18ed9a3daf4ee497871d8a3ab63d0cba4cd9c67b4d515da591dbc0bad01e6"
Jul 16 16:15:43.518984 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.518940    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5d18ed9a3daf4ee497871d8a3ab63d0cba4cd9c67b4d515da591dbc0bad01e6"} err="failed to get container status \"c5d18ed9a3daf4ee497871d8a3ab63d0cba4cd9c67b4d515da591dbc0bad01e6\": rpc error: code = NotFound desc = could not find container \"c5d18ed9a3daf4ee497871d8a3ab63d0cba4cd9c67b4d515da591dbc0bad01e6\": container with ID starting with c5d18ed9a3daf4ee497871d8a3ab63d0cba4cd9c67b4d515da591dbc0bad01e6 not found: ID does not exist"
Jul 16 16:15:43.526946 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.526911    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"]
Jul 16 16:15:43.532820 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.532798    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-590f9a-openshift-default-5fcb86f8f5-dr2kk"]
Jul 16 16:15:43.753782 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:43.753701    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qj26h_df6431f6-77da-4f5d-8330-ec9a1a08d6ab/manager/0.log"
Jul 16 16:15:44.140650 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:44.140621    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-kzl5p_fafcdc9c-b525-4453-b491-839385024054/server/0.log"
Jul 16 16:15:44.266627 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:44.266595    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2p5gp_3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a/manager/0.log"
Jul 16 16:15:44.268718 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:44.268692    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b99de390-ef87-4649-b524-32eb2f343632" path="/var/lib/kubelet/pods/b99de390-ef87-4649-b524-32eb2f343632/volumes"
Jul 16 16:15:44.692645 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:44.692617    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b5b94d4fb-qjzng_0a7674a4-071f-4241-9adf-e3fd7567f5de/copy-manifests/0.log"
Jul 16 16:15:44.702023 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:44.701998    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b5b94d4fb-qjzng_0a7674a4-071f-4241-9adf-e3fd7567f5de/manager/0.log"
Jul 16 16:15:45.598240 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:45.598214    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-swmwp_7b36b36c-f9a0-4d6a-9311-c17ab19ffd7b/postgres/0.log"
Jul 16 16:15:47.126526 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:47.126489    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9lgck_a7a0d14c-6f16-4d9a-929a-3695870b25e2/manager/0.log"
Jul 16 16:15:47.254713 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:47.254686    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-l24fv_46ec8fae-a977-4a40-94d9-a83ab2c75916/manager/0.log"
Jul 16 16:15:47.380056 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:47.379978    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-vnp4r_e3fd0bb7-775a-456d-83bf-4af1b7d8c437/kuadrant-console-plugin/0.log"
Jul 16 16:15:47.526779 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:47.526748    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-85qw6_a600d138-c9a7-4b42-b67b-7ebb67fcb59a/registry-server/0.log"
Jul 16 16:15:47.931692 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:47.931665    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vzk4t_24eda07a-6e1a-4f20-8fe9-f7815cc639f1/manager/0.log"
Jul 16 16:15:48.294193 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:48.294094    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47m88r_e3b4d2a6-fa24-4c3a-898a-7e35613dac00/istio-proxy/0.log"
Jul 16 16:15:48.660573 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:48.660545    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bb6d5565c-lk4jh_7b853572-837c-44c8-bd2b-9a9919a6ca98/kube-auth-proxy/0.log"
Jul 16 16:15:48.782110 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:48.782076    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rcsbl_ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c/istio-proxy/1.log"
Jul 16 16:15:48.919841 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:48.919742    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-kpmn2_a82ac839-3e06-4cae-b183-f2ac59856423/payload-pre-processing/0.log"
Jul 16 16:15:49.047085 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:49.044466    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-84dwr_1c89cea7-abe8-489e-86ef-0d979a20d8db/payload-processing/0.log"
Jul 16 16:15:49.536019 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:49.535985    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-9r4tk_6322f4a6-f0f2-494c-a83e-b59ecb5eac30/main/0.log"
Jul 16 16:15:49.780748 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:49.780720    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-qzs5h_67545190-9949-4ed0-a636-2f34f144d73c/main/0.log"
Jul 16 16:15:50.144740 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:50.144710    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-xvqp4_4689b43c-42fd-418d-866b-945e55bb5d66/main/0.log"
Jul 16 16:15:51.951588 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:51.951549    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:54.171227 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:54.171184    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:54.229714 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:54.229675    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log" limit=1024
Jul 16 16:15:57.051814 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:57.051719    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4t8xm_83adbeaf-0de4-4fec-be65-5a9a8b9ce326/global-pull-secret-syncer/0.log"
Jul 16 16:15:57.266368 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:57.266335    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b4cff_51bd5dc0-5c9d-4c3f-b95b-1e275a3f8730/konnectivity-agent/0.log"
Jul 16 16:15:57.424073 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:15:57.424037    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-84.ec2.internal_391d2eed45bf7533effc099da20934d8/haproxy/0.log"
Jul 16 16:16:02.528379 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:02.528345    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9lgck_a7a0d14c-6f16-4d9a-929a-3695870b25e2/manager/0.log"
Jul 16 16:16:02.563682 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:02.563653    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-l24fv_46ec8fae-a977-4a40-94d9-a83ab2c75916/manager/0.log"
Jul 16 16:16:02.595086 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:02.595058    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-vnp4r_e3fd0bb7-775a-456d-83bf-4af1b7d8c437/kuadrant-console-plugin/0.log"
Jul 16 16:16:02.657861 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:02.657830    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-85qw6_a600d138-c9a7-4b42-b67b-7ebb67fcb59a/registry-server/0.log"
Jul 16 16:16:02.949547 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:02.949504    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vzk4t_24eda07a-6e1a-4f20-8fe9-f7815cc639f1/manager/0.log"
Jul 16 16:16:05.361476 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:05.361344    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bkkkh_09f00530-b743-432e-a12f-af9301dbd46a/node-exporter/0.log"
Jul 16 16:16:05.385548 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:05.385520    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bkkkh_09f00530-b743-432e-a12f-af9301dbd46a/kube-rbac-proxy/0.log"
Jul 16 16:16:05.413628 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:05.413600    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bkkkh_09f00530-b743-432e-a12f-af9301dbd46a/init-textfile/0.log"
Jul 16 16:16:06.695245 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.695208    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"]
Jul 16 16:16:06.695734 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.695720    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b99de390-ef87-4649-b524-32eb2f343632" containerName="istio-proxy"
Jul 16 16:16:06.695810 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.695739    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b99de390-ef87-4649-b524-32eb2f343632" containerName="istio-proxy"
Jul 16 16:16:06.695810 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.695764    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerName="istio-proxy"
Jul 16 16:16:06.695810 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.695773    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerName="istio-proxy"
Jul 16 16:16:06.695810 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.695796    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50934403-ff3d-4f59-a08d-221a3456e9c1" containerName="main"
Jul 16 16:16:06.695810 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.695805    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="50934403-ff3d-4f59-a08d-221a3456e9c1" containerName="main"
Jul 16 16:16:06.696099 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.695819    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerName="istio-proxy"
Jul 16 16:16:06.696099 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.695827    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerName="istio-proxy"
Jul 16 16:16:06.696099 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.695911    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerName="istio-proxy"
Jul 16 16:16:06.696099 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.695925    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b99de390-ef87-4649-b524-32eb2f343632" containerName="istio-proxy"
Jul 16 16:16:06.696099 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.695953    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b99de390-ef87-4649-b524-32eb2f343632" containerName="istio-proxy"
Jul 16 16:16:06.696099 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.695963    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc67b0d3-1f45-4971-81ad-214aa8c1b844" containerName="istio-proxy"
Jul 16 16:16:06.696099 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.695976    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="50934403-ff3d-4f59-a08d-221a3456e9c1" containerName="main"
Jul 16 16:16:06.700451 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.700427    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.703067 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.703043    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dnqzv\"/\"openshift-service-ca.crt\""
Jul 16 16:16:06.703191 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.703125    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dnqzv\"/\"kube-root-ca.crt\""
Jul 16 16:16:06.703309 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.703294    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dnqzv\"/\"default-dockercfg-5jg58\""
Jul 16 16:16:06.707622 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.707218    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"]
Jul 16 16:16:06.824872 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.824835    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba85f994-5f67-48f2-92c8-7193fabb97be-lib-modules\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.825074 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.824899    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba85f994-5f67-48f2-92c8-7193fabb97be-podres\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.825074 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.824982    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba85f994-5f67-48f2-92c8-7193fabb97be-sys\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.825074 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.825022    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba85f994-5f67-48f2-92c8-7193fabb97be-proc\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.825074 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.825058    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55k8k\" (UniqueName: \"kubernetes.io/projected/ba85f994-5f67-48f2-92c8-7193fabb97be-kube-api-access-55k8k\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.925873 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.925833    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba85f994-5f67-48f2-92c8-7193fabb97be-sys\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.925873 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.925882    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba85f994-5f67-48f2-92c8-7193fabb97be-proc\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.926157 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.925910    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55k8k\" (UniqueName: \"kubernetes.io/projected/ba85f994-5f67-48f2-92c8-7193fabb97be-kube-api-access-55k8k\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.926157 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.925986    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba85f994-5f67-48f2-92c8-7193fabb97be-sys\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.926157 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.925997    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba85f994-5f67-48f2-92c8-7193fabb97be-proc\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.926157 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.926002    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba85f994-5f67-48f2-92c8-7193fabb97be-lib-modules\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.926157 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.926085    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba85f994-5f67-48f2-92c8-7193fabb97be-podres\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.926157 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.926121    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba85f994-5f67-48f2-92c8-7193fabb97be-lib-modules\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.926453 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.926227    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba85f994-5f67-48f2-92c8-7193fabb97be-podres\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:06.935547 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:06.935525    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55k8k\" (UniqueName: \"kubernetes.io/projected/ba85f994-5f67-48f2-92c8-7193fabb97be-kube-api-access-55k8k\") pod \"perf-node-gather-daemonset-spm2q\" (UID: \"ba85f994-5f67-48f2-92c8-7193fabb97be\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:07.011507 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:07.011406    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:07.150040 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:07.150009    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"]
Jul 16 16:16:07.152988 ip-10-0-141-84 kubenswrapper[2570]: W0716 16:16:07.152917    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podba85f994_5f67_48f2_92c8_7193fabb97be.slice/crio-c17574ef8d965d9b0552fa33cf85bacf3ac58da97220b000e3e677ae7ee56363 WatchSource:0}: Error finding container c17574ef8d965d9b0552fa33cf85bacf3ac58da97220b000e3e677ae7ee56363: Status 404 returned error can't find the container with id c17574ef8d965d9b0552fa33cf85bacf3ac58da97220b000e3e677ae7ee56363
Jul 16 16:16:07.306097 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:07.306012    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-csgsh_56ee72e4-c4b3-4042-bb36-d5b239671b10/networking-console-plugin/0.log"
Jul 16 16:16:07.621362 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:07.621320    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q" event={"ID":"ba85f994-5f67-48f2-92c8-7193fabb97be","Type":"ContainerStarted","Data":"e6555c2fb66e76f3c4b60be7473586850741b2ae19d6845412cd2ccc821a077c"}
Jul 16 16:16:07.621568 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:07.621400    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q" event={"ID":"ba85f994-5f67-48f2-92c8-7193fabb97be","Type":"ContainerStarted","Data":"c17574ef8d965d9b0552fa33cf85bacf3ac58da97220b000e3e677ae7ee56363"}
Jul 16 16:16:07.621568 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:07.621471    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:07.642063 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:07.642010    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q" podStartSLOduration=1.641996563 podStartE2EDuration="1.641996563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:16:07.640093198 +0000 UTC m=+2433.828957153" watchObservedRunningTime="2026-07-16 16:16:07.641996563 +0000 UTC m=+2433.830860518"
Jul 16 16:16:10.227790 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:10.227759    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-77d2v_83c60a71-04d9-4ee3-8431-b381d610783c/dns/0.log"
Jul 16 16:16:10.271134 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:10.271104    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-77d2v_83c60a71-04d9-4ee3-8431-b381d610783c/kube-rbac-proxy/0.log"
Jul 16 16:16:10.452809 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:10.452777    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kqh5g_6a32f024-c920-4db1-8c81-37477f8dc910/dns-node-resolver/0.log"
Jul 16 16:16:11.057989 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:11.057953    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-756bbccf75-r2xtx_6721b192-01a8-44b8-ac19-b98d931a02f4/registry/0.log"
Jul 16 16:16:11.194360 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:11.194331    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ftrpc_764123aa-64ce-40d5-9d5f-8be36c2026f6/node-ca/0.log"
Jul 16 16:16:12.183207 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:12.183177    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47m88r_e3b4d2a6-fa24-4c3a-898a-7e35613dac00/istio-proxy/0.log"
Jul 16 16:16:12.351909 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:12.351877    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bb6d5565c-lk4jh_7b853572-837c-44c8-bd2b-9a9919a6ca98/kube-auth-proxy/0.log"
Jul 16 16:16:12.389701 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:12.389664    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rcsbl_ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c/istio-proxy/0.log"
Jul 16 16:16:12.447273 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:12.447188    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rcsbl_ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c/istio-proxy/1.log"
Jul 16 16:16:12.508079 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:12.508044    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-kpmn2_a82ac839-3e06-4cae-b183-f2ac59856423/payload-pre-processing/0.log"
Jul 16 16:16:12.584481 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:12.584430    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-84dwr_1c89cea7-abe8-489e-86ef-0d979a20d8db/payload-processing/0.log"
Jul 16 16:16:13.335529 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:13.335496    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9bzgd_56a2820e-a5d6-43e8-ad74-3fa061bd5071/serve-healthcheck-canary/0.log"
Jul 16 16:16:13.636349 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:13.636277    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-spm2q"
Jul 16 16:16:13.930383 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:13.930288    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b9qf6_b7607d9f-e61a-4b41-9494-4622fb17f848/kube-rbac-proxy/0.log"
Jul 16 16:16:13.958048 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:13.958021    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b9qf6_b7607d9f-e61a-4b41-9494-4622fb17f848/exporter/0.log"
Jul 16 16:16:13.983464 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:13.983425    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b9qf6_b7607d9f-e61a-4b41-9494-4622fb17f848/extractor/0.log"
Jul 16 16:16:16.742338 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:16.742221    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qj26h_df6431f6-77da-4f5d-8330-ec9a1a08d6ab/manager/0.log"
Jul 16 16:16:17.136856 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:17.136820    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-kzl5p_fafcdc9c-b525-4453-b491-839385024054/server/0.log"
Jul 16 16:16:17.300828 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:17.300789    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2p5gp_3ecda9dd-6d1a-49d3-819b-873d7b7b0e5a/manager/0.log"
Jul 16 16:16:17.680288 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:17.680253    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b5b94d4fb-qjzng_0a7674a4-071f-4241-9adf-e3fd7567f5de/manager/0.log"
Jul 16 16:16:17.703679 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:17.703653    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b5b94d4fb-qjzng_0a7674a4-071f-4241-9adf-e3fd7567f5de/copy-manifests/0.log"
Jul 16 16:16:17.820248 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:17.820218    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47m88r_e3b4d2a6-fa24-4c3a-898a-7e35613dac00/istio-proxy/0.log"
Jul 16 16:16:17.959898 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:17.959864    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bb6d5565c-lk4jh_7b853572-837c-44c8-bd2b-9a9919a6ca98/kube-auth-proxy/0.log"
Jul 16 16:16:17.980418 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:17.980383    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rcsbl_ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c/istio-proxy/0.log"
Jul 16 16:16:18.033895 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:18.033865    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rcsbl_ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c/istio-proxy/1.log"
Jul 16 16:16:18.090811 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:18.090775    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-kpmn2_a82ac839-3e06-4cae-b183-f2ac59856423/payload-pre-processing/0.log"
Jul 16 16:16:18.158491 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:18.158455    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-84dwr_1c89cea7-abe8-489e-86ef-0d979a20d8db/payload-processing/0.log"
Jul 16 16:16:19.358805 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:19.358774    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47m88r_e3b4d2a6-fa24-4c3a-898a-7e35613dac00/istio-proxy/0.log"
Jul 16 16:16:19.494768 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:19.494737    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bb6d5565c-lk4jh_7b853572-837c-44c8-bd2b-9a9919a6ca98/kube-auth-proxy/0.log"
Jul 16 16:16:19.519997 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:19.519954    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rcsbl_ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c/istio-proxy/0.log"
Jul 16 16:16:19.582487 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:19.582421    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rcsbl_ec8e2873-e0cc-4395-b2c9-2ae1f99c4c1c/istio-proxy/1.log"
Jul 16 16:16:19.637121 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:19.637033    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-kpmn2_a82ac839-3e06-4cae-b183-f2ac59856423/payload-pre-processing/0.log"
Jul 16 16:16:19.708067 ip-10-0-141-84 kubenswrapper[2570]: I0716 16:16:19.708020    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-84dwr_1c89cea7-abe8-489e-86ef-0d979a20d8db/payload-processing/0.log"